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Author’s Preface 


This volume pulls together and republishes, with some editing, 
updating, and additions, articles written during 1978-86 for internal 
use within the CIA Directorate of Intelligence. Four of the articles 
also appeared in the Intelligence Community journal Studies in Intelli- 
gence during that time frame. The information is relatively timeless 
and still relevant to the never-ending quest for better analysis. 

The articles are based on reviewing cognitive psychology litera- 
ture concerning how people process information to make judgments 
on incomplete and ambiguous information. I selected the experiments 
and findings that seem most relevant to intelligence analysis and most 
in need of communication to intelligence analysts. I then translated 
the technical reports into language that intelligence analysts can 
understand and interpreted the relevance of these findings to the prob- 
lems intelligence analysts face. 

The result is a compromise that may not be wholly satisfactory 
to either research psychologists or intelligence analysts. Cognitive 
psychologists and decision analysts may complain of oversimplifica- 
tion, while the non-psychologist reader may have to absorb some new 
terminology. Unfortunately, mental processes are so complex that dis- 
cussion of them does require some specialized vocabulary. Intelli- 
gence analysts who have read and thought seriously about the nature 
of their craft should have no difficulty with this book. Those who are 
plowing virgin ground may require serious effort. 

I wish to thank all those who contributed comments and sugges- 
tions on the draft of this book: Jack Davis (who also wrote the Intro- 
duction); four former Directorate of Intelligence (DI) analysts whose 
names cannot be cited here; my current colleague, Prof. Theodore 
Sarbin; and my editor at the CIA’s Center for the Study of Intelli- 
gence, Hank Appelbaum. All made many substantive and editorial 
suggestions that helped greatly to make this a better book. 


—Richards J. Heuer, Jr. 


Foreword 


By Douglas MacEachin' 


My first exposure to Dick Heuer’s work was about 18 years ago, 
and I have never forgotten the strong impression it made on me then. 
That was at about the midpoint in my own career as an intelligence 
analyst. After another decade and a half of experience, and the oppor- 
tunity during the last few years to study many historical cases with the 
benefit of archival materials from the former USSR and Warsaw Pact 
regimes, reading Heuer’s iatest preseniation has had even more reso- 
nance. 

| know from first-hand encounters that rsany CIA officers tend to 
react skeptically to treatises on analytic epistemology. This is under- 
standable. Too often, such treatises end up prescribing models as 
answers to the problem. These models seem to have little practical 
value to intelligence analysis, which takes place not in a seminar but 
rather in a fast-breaking world of policy. But that is not the main prob- 
lem Heuer is addressing. 

What Heuer examines so clearly and effectively is how the 
human thought process builds its own models through which we pro- 
cess information. This is not a phenomenon unique to intelligence; as 
Heuer’s research demonstrates, it is part of the natural functioning of 
the human cogaitive process, and it has been demonstrated across a 
broad range of fields ranging from medicine to stock market analysis. 

The process of analysis itself reinforces this natural function of 
the human brain. Analysis usually involves creating models, even 
though they may not be labeled as such. We set forth certain under- 
Standings and expectations about cause-and-effect relationships and 
then process and interpret information through these models or filters. 

The discussion in Chapter 5 on the limits to the value of addi- 
tional information deserves special attention, in my view—particu- 
larly for an intelligence organization. What it illustrates is that too 


Douglas MacEachin is a iormer CIA Deputy Director of Intelligence. After 32 years with the 
Agency, he retired in 1997 and became a Senior Fellow at Harvard University’s John F. Kennedy 
School of Government. 


often, newly acquired information is evaluated and processed through 
the existing analytic model, rather than being used to reassess the 
premises of the model itself. The detrimental effects of this natural 
human tendency stem from the raison d’etre of an organization cre- 
ated to acquire special, critical information available only through 
covert means, and to produce analysis integrating this special informa- 
tion with the total knowledge base. 

1 doubt that any veteran intelligence officer will be able to read 
this book without recalling cases in which the mental processes 
described by Heuer have had an adverse impact on the quality of anal- 
ysis. How many times have we encountered situations in which com- 
pletely plausible premises, based on solid expertise, have been used to 
construct a logically valid forecast—with virtually unanimous agree- 
ment-——that turned out to be dead wrong? In how many of these 
instances have we determined, with hindsight, that the problem was 
not in the logic but in the fact that one of the premises——however 
plausible it seemed at the time—was incorrect? In how many of these 
instances have we becn forced to admit that the erroneous premise 
was not empirically based but rather a conclusion developed from its 
own model (sometimes called an assumption)? And in how many 
cases was it determined after the fact that information had been avail- 
able which should have provided a basis for questioning one or more 
premises, and that a change of the relevant premise(s) would have 
changed the analytic model and pointed to a different outcome? 

The commonly prescribed remedy for shortcomings in intelli- 
gence analysis and estimates—most vociferously after intelligence 
“failures”’—is a major increase in expertise. Heuer’s research and the 
studies he cites pose a serious challenge to that conventional wisdom. 
The data show that expertise itself is no protection from the common 
analytic pitfalls that are endemic to the human thought process. This 
point has been demonstrated in many fields beside intelligence analy- 
Sis. 

A review of notorious intelligence failures demonstrates that the 
analytic traps caught the experts as much as anybody. Indeed, the data 
show that when experts fall victim to these traps, the effects can be 
aggravated by the confidence that attaches to expertise—both in their 
own view and in the perception of others. 

These observations should in no way be construed as a denigra- 
tion of the value of expertise. On the contrary, my own 30-plus years 
in the business of intelligence analysis biased me in favor of the view 
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that, endless warnings of information overload notwithstanding, there 
is no such thing as too much information or expertise. And my own 
observations of CIA analysts sitting at the same table with publicly 
renowned experts have given me great confidence that attacks on the 
expertise issue are grossly misplaced. The main difference is that one 
group gets to promote its reputations in journals, while the other 
works in a closed environment in which the main readers are members 
of the intelligence world’s most challenging audience—te policy- 
making community. 

The message that comes through in Heuer’s presentation is that 
information and expertise are a necessary but not sufficient means of 
making intelligence analysis the special product that it needs to be. A 
comparable effort has to be devoted to the science of analysis. This 
effort has to start with a clear understanding of the inherent strengths 
and weaknesses of the primary analytic mechanism—the human 
mind—and the way it processes information. 

I believe there is a significant cultural element in how intelli- 
gence analysts define themselves: Are we substantive experts 
employed by CIA, or are we professional analysts and intelligence 
officers whose expertise lies in our ability to adapt quickly to diverse 
issues and problems and analyze them effectively? In the world at 
large, substantive expertise is far more abundant than expertise on 
analytic science and the human mental processing of information. 
Dick Heuer makes clear that the pitfalls the human mental process 
sets for analysts cannot be eliminated; they are part of us. What can 
be done is to train people how to look for and recognize these mental 
obstacles, and how to develop procedures designed to 0% set them. 

Given the centrality of analytic science for the intelligence mis- 
sion, a key question that Heuer’s book poses is: Compared with other 
areas of our business, have we committed a commensurate effort to 
the study of analytic science as a p: fessional requirement? How do 
the effort and resource commitments in this area compare to, for 
example, the effort and commitment to the development of analysts’ 
writing skills? 

Heuer’s book does not pretend to be the last word on this issue. 
Hopefully, it will be a stimulant for much more work. 


Introduction 


Improving Intelligence Analysis 
at CIA: Dick Heuer’s Contribution 
to Intelligence Analysis 


by jack Davis' 


1 applaud CIA’s Center for the Study of Intelligence for making 
the work of Richards J. Heuer, Jr. on the psychology of intelligence 
analysis available to a new generation of intelligence practitioners and 
scholars. 

Dick Heuer’s ideas on how to improve analysis focus on helping 
analysts compensate for the human mind’s limitations in dealing with 
complex problems that typically involve ambiguous information, mul- 
tiple players, and fluid circumstances. Such multi-faceted estimative 
challenges have proliferated in the turbulent post-Cold War world. 

Heuer’s message to analysts can be encapsulated by quoting two 
sentences from Chapter 4 of this book: 


Intelligence analysts should be self-conscious about their 
reasoning processes. They should think about how they 
make judgments and reach conclusions, aot just about the 
judgments and conclusions themselves. 


Heuer’s ideas are applicable to any analytical endeavor. In this 
Introduction, | have concentrated on his impact—and that of other 
pioneer thinkers in the intelligence analysis field—at CIA, because 
that is the institution that Heuer and his predecessors, and I myself, 
know best, having spent the bulk of our intelligence careers there. 


Jack Davis served with the Directorate of Intelligence (DI), the National Intelligence Council, and 
the Office of Training during his CIA career. He is nov an independent contractor who specializes 
in devel-ping and teaching analytic tradecraft. Among his publications is Uncertainty, Surprise, 
and Warning (1996). 
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Leading Contributors to Quality of Analysis 


Intelligence analysts, in seeking to make sound judgments, are 
always under challenge from the complexities of the issues they 
address and from the demands made on them for timeliness and vol- 
ume of production. Four Agency individuals over the decades stand 
out for having made major contributions on how to deal with these 
challenges to the quality of analysis. 

My short list of the people who have had the greatest positive 
impact on CIA analysis consists of Sherman Kent, Robert Gates, Dou- 
glas MacEachin, and Richards Heuer. My selection methodology was 
simple. | asked myself: Whose insights have influenced me the most 
during my four decades of practicing, teaching, and writing about 
analysis? 


Sherman Kent 

Sherman Kent's pathbreaking contributions to analysis cannot be 
dene justice in a couple of paragraphs, and | refer readers to fuller 
treatments elsewhere.’ Here | address his general legacy to the analyti- 
cal profession. 

Kent, a professor of European history at Yale, worked in the 
Research and Analysis branch of the Office of Strategic Services dur- 
ing World War Il. He wrote an influential book, Strategic Intelligence 
for American World Power, while at the National War College in the 
late 1940s. He served as Vice Chairman and then as Chairman of the 
DCI's Board of National Estimates from 1950 to 1967. 

Kent's greatest contribution to the quality of analysis was to 
define an honorable place for the analyst—the thoughtful individual 
“applying the instruments of reason and the scientific method”—in an 
intelligence world then as now dominated by collectors and ope: ators. 
In a second (1965) edition of Strategic Intelligence, Kent took ac. sunt 
of the coming computer age as well as human and technical collec- 
tors in proclaiming the centrality of the analyst: 


See, in particular, the editor's unclassified introductory essay and “Tribute” by Harold P. Ford in 
Donald P. Steury, Sherman Kent ard the Board of National Estimates: Collected Essays (CVA, 
Center for the Study of Intelligence, 1994). Hereinafter cited as Steury, Kent 


Whatever the complexities of the puzzles we strive to solve 
and whatever the sophisticated techniques we may use to 
collect the pieces and store them, there can never be a time 
when the thoughtful man can be supplanted as the intelli- 
gence device supreme. 


More specifically, Kent advocated application of the techniques 
of “scientific” study of the past to analysis of complex ongoing situa- 
tions and estimates of likely future events. Just as rigorous “impar- 
tial” analysis could cut through the gaps and ambiguities of 
information on events long past and point to the most probable expla- 
nation, he contended, the powers of the critical mind could turn to 
events that had not yet transpired to determine the most probable 
developments. ' 

To this end, Kent developed the concept of the analytic pyramid, 
featuring a wide base of factual information and sides comprised of 
sound assumptions, which pointed to the nost likely future scenario at 
the apex.* 

In his proselytizing and in practice, Kent battled against bureau- 
cratic and ideological biases, which he recognized as impediments to 
sound analysis, and against imprecise estimative terms that he saw as 
obstacles to conveying clear messages to readers. Although he was 
aware of what is now called cognitive bias, his writings urge analysts 
to “make the call” without much discussion of how limitations of the 
human mind were to be overcome. 

Not many Agency analysts read Kent nowadays. But he had a 
profound impact on earlier generations of analysts and managers, and 
his work continues to exert an indirect influence among practitioners 
of the analytic profession. 


Sherman Kent, Writing History, second edition (1967). The first edition was published in 1941. 
when Keni was an assistant professor of history at Yale. In the first chapter, “Why History.” he pre- 
sented ideas and recommendations that he later adapted for intelligence analysis 
’ Kent, “Estimates and Influence” (1968), in Steury, Kent 
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Robert Gates 
Bob Gates served as Deputy Director of Central Intelligence 


(1986-1989) and as DCI (1991-1993). But his greatest impact on the 
quality of CIA analysis came during his 1982-1986 stint as Deputy 
Director for Intelligence (DDI). 

Initially schooled as a political scientist, Gates earned a Ph.D. in 
Soviet studies at Georgetown while working as an analyst at CIA. As 
a member of the National Security Council staff during the 1970s, he 
gained invaluable insight into how policymakers use intelligence anal- 
ysis. Highly intelligent, exceptionally hard-working, and skilled in the 
bureaucratic arts, Gates was appointed DDI by DCI William Casey in 
good part because he was one of the few insiders Casey found who 
shared the DCI’s views on what Casey saw as glaring deficiencies of 
Agency analysts.‘ Few analysts and managers who heard it have for- 
gotten Gates’ blistering criticism of analytic performance in his 1982 
“inaugural” speech as DDI. 

Most of the public commentary on Gate. and Agency analysis 
concerned charges of politicization levied against him, and his defense 
against such charges, during Senate hearings for his 1991 confirma- 
tion as DCI. The heat of this debate was slow to dissipate among CIA 
analysts, as reflected in the pages of Studies in Intelligence, the 
Agency journal founded by Sherman Kent in the 1950s.° 

I know of no written retrospective on Gates’ contribution to 
Agency analysis. My insights into his ideas about analysis came 
mostly through an arms-length collaboration in setting up and run- 
ning an Agency training course entitled “Seminar on Intelligence Suc- 
cesses and Failures.’ During his tenure as DDI, only rarely could you 
hold a conversation with analysts or managers without picking up 
additional viewpoints, thoughtful and otherwise, on what Gates was 
doing to change CIA analysis. 

Gates’s ideas for overcoming what he saw as insular, flabby, and 
inccherent argumentation featured the importance of distinguishing 
between what analysts know and what they believe—that is, to make 


* Casey, very early in his tenure as DCI (1981-1987), opined to me that the trouble with Agency 
analysts is that they went from sitting on their rear ends at universities to sitting on their rear ends 
at CIA, without seeing the real world. 

* “The Gates Hearings: Politicization and Soviet Analysis at CIA”, Studies in Intelligence (Spring 
1994). “Communication to the Editor: The Gates Hearings: A Biased Account,” Studies in Intelli- 
gence (Fall 1994). 

DCI Casey requested that the Agency's training office provide this seminar so that, at the least, 
analysts could learn from their own mistakes. DDI Gates carefully reviewed the statement of goals 
for the seminar, the outline of course units, and the required reading list. 
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clear what is “fact” (or reliably reported information) and what is the 
analyst's opinion (which had to be persuasively supported w ._/i- 
dence). Among his other tenets were the need to seek the views of 
non-CIA experts, including academic specialists and policy officials, 
and to present alternate future scenarios. 

Gates’s main impact, though, came from practice—from his 
direct involvement in implementing his ideas. Using his authority as 
DDI, he reviewed critically almost all in-depth assessments and cur- 
rent intelligence articles prior to publication. With help from his dep- 
uty and two rotating assistants from the ranks of rising junior 
managers, Gates raised the standards for DDI review dramatically—in 
essence, from “looks good to me” to “show me your evidence.” 

As the many drafts Gates rejected were sent back to managers 
who had approved tiiem—accompanied by the DDI’s comments about 
inconsistency, lack of clarity, substantive bias, and poorly supported 
judgments—the whole chain of review became much more rigorous. 
Analysts and their managers raised their standards to avoid the pain of 
DDI rejection. Both career advancement and ego were at stake. 

The rapid and sharp increase in attention paid by analysts and 
managers to the underpinnings for their substantive judgments proba- 
bly was without precedent in the Agency’s history. The longer term 
benefits of the intensified review process were more limited, however, 
because insufficient atteation was given to clarifying tradecraft prac- 
tices that would promote analytic soundness. More than one partici- 
pant in the process observed that a lack of guidelines for meeting 
Gates’s standards led to a large amount of “wheel-spinning.” 

Gates’s impact, like Kent’s, has to be seen on two planes. On the 
one hand, little that Gates wrote on the craft of analysis is read these 
days. But even though his pre-publication review process was discon- 
tinued under his successors, an enduring awareness of his standards 
still gives pause at jumping to conclusions to many managers and ana- 
lysts who experienced his criticism first-hand. 


Douglas MacEachin 

Doug MacEachin, DDI from 1993 to 1996, sought to provide an 
essential ingredient for ensuring implementation of sound analytic 
standards: corporate tradecraft standards for analysts. This new trade- 
craft was aimed in particular at ensuring that sufficient attention 
would be paid to cognitive challenges in assessing complex issues. 
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MacEachin set out his views on Agency analytical faults and 
correctives in The Tradecraft of Analysis: Challenge and Change in 
the CIA.* My commentary on his contributions to sound analysis is 
also informed by a series of exchanges with him in 1994 and 1995. 

MacEachin’s university major was economics, but he also 
showed great interest in philosophy. His Agency career—like 
Gates’—included an extended assignment to a policymaking office. 
‘He came away from this experience with new insights on what consti- 
tutes “value-added” intelligence usable by policymakers. Subse- 
quently, as CIA’s senior manager on arms control issues, he dealt 
regularly with a cadre of tough-minded policy officials who let him 
know in blunt terms what worked as effective policy support and what 
did not. 

By the time MacEachin became DDI in 1993, Gates’s policy of 
DDI front-office pre-publication review of nearly all DI analytical 
studies had been discontinued. MacEachin took a different approach; 
he read—mostly on weekends—and reflected on numerous already- 
published DI analytical papers. He did not like what he found. In his 
words, roughly a third of the papers meant to assist the policymaking 
process had no discernible argumentation to bolster the credibility of 
intelligence judgments, and another third suffered from flawed argu- 
mentation. This experience, along with pressures on CIA for better 
analytic performance in the wake of alleged “intelligence failures” 
concerning Iraq’s invasion of Kuwait, prompted his decision to launch 
a major new effort to raise analytical standards.’ 

MacEachin advocated an approach to structured argumentation 
called “linchpin analysis,” to which he contributed muscular terms 
designed to overcome many CIA professionals’ distaste for academic 
nomenclature. The standard academic term “key variables” became 
drivers. “Hypotheses” concerning drivers became linchpins—assump- 
tions underlying the argument—and these had to be explicitly spelled 
out. MacEachin also urged that greater attention be paid to analytical 
processes for alerting policymakers to changes in circumstances that 
would increase the likelihood of alternative scenarios. 

MacEachin thus worked to put in place systematic and transpar- 
ent standards for determining whether analysts had met their responsi- 
bilities for critical thinking. To spread understanding and application 
of the standards, he mandated creation of workshops on linchpin 


‘Unclassified paper published in 1994 by the Working Group on Intelligence Reform, which had 
been created in 1992 by the Consortium for the Study of Intelligence, Washington, DC. 
’ Discussion between MacEachin and the author of this Introduction, 1994. 


XVI 


analysis for managers and production of a series of notes on analytical 
tradecraft. He also directed that the DI’s performance on tradecraft 
standards be tracked and that recognition be given to exemplary 
assessments. Perhaps most ambitious, he saw to it that instruction on 
standards for analysis was incorporated into a new training course, 
“Tradecraft 2000." Nearly all DI managers and analysts attended this 
course during 1996-97. 

As of this writing (early 1999), the long-term staying power of 
MacEachin’s tradecraft initiatives is not yet clear. But much of what 
he advocated has endured so far. Many DI analysts use variations on 
his linchpin concept to produce soundly argued forecasts. In the train- 
ing realm, “Tradecrait 2000” has been supplanted by a new course 
that teaches the same concepts to newer analysts. But examples of 
what MacEachin would label as poorly substantiated analysis are still 
seen. Clearly, ongoing vigilance is needed to keep such analysis from 
finding its way into DI products. 


Richards Heuer 

Dick Heuer was—and is—much less well known within the CIA 
than Kent, Gates, and MacEachin. He has not received the wide 
acclaim that Kent enjoyed as the father of professional analysis, and 
he has lacked the bureaucratic powers that Gates and MacEachin 
could wield as DDIs. But his impact on the quality of Agency analysis 
arguably has been at least as important as theirs. 

Heuer received a degree in philosophy in 1950 from Williams 
College, where, he notes, he became fascinated with the fundamental 
epistemological question, “What is truth and how can we know it?” In 
1951, while a graduate student at the University of California’s Berke- 
ley campus, he was recruited as part of the CIA’s buildup during the 
Korean War. The recruiter was Richard Helms, OSS veteran and ris- 
ing player in the Agency’s clandestine service. Future DCI Helms, 
according to Heuer, was looking for candidates for CIA employment 
among recent graduates of Williams College, his own alma mater. 
Heuer had an added advantage as a former editor of the college’s 
newspaper, a position Helms had held some 15 years earlier. '° 


Letter to the author of this Introduction, 1998. 
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In 1975, after 24 years in the Directorate of Operations, Heuer 
moved to the DI. His earlier academic interest in how we know the 
truth was rekindled by two experiences. One was his involvement in 
the controversial case of Soviet KGB defector Yuriy Nosenko. The 
other was learning new approaches to social science methodology 
while earning a Master's degree in international relations at the Uni- 
versity of Southern California’s European campus. 

At the time he retired in 1979, Heuer headed the methodology 
unit in the DI’s political analysis office. He originally prepared most 
of the chapters in this book as individual articles between 1978 and 
1986; many of them were written for the DI after his retirement. He 
has updated the articles and prepared some new material for inclusion 
in this book. 


Heuer’s Central Ideas 


Dick Heuer’s writings make three fundamental points about the 
cognitive challenges intelligence analysts face: 


¢ The mind is poorly “wired” to deal effectively with both inher- 
ent uncertainty (the natural fog surrounding complex, indeter- 
minate intelligence issues) and induced uncertainty (the man- 
made fog fabricated by denial and deception operations). 


Even increased awareness of cognitive and other “unmotivated” 
biases, such as the tendency to see information confirming an 
already-held judgment more vividly than one sees “disconfirm- 
ing” information, does little by itself to help analysts deal effec- 
tively with uncertainty. 


Tools and techniques that gear the analyst’s mind to apply 
higher levels of critical thinking can substantially improve anal- 
ysis on complex issues on which information is incomplete, 
ambiguous, and often deliberately distorted. Key examples of 
such intellectual devices include techniques for structuring 
information, challenging assumptions, and exploring alterna- 
tive interpretations. 


The following passage from Heuer’s 1980 article entitled “Per- 
ception: Why Can’t We See What Is There to be Seen?” shows that 
his ideas were similar to or compatible with MacEachin’s concepts of 
linchpin analysis. 


Given the difficulties inherent in the human processing of 
complex information, a prudent mz agement system 
should: 


¢ Encourage products that (a) clearly delineate their 
assumptions and chains of inference and (b) specify 
the degree and source of the uncertainty involved in 
the conclusions. 


¢ Emphasize procedures that expose and elaborate 
alternative points of view—analytic debates, devil’s 
advocates, interdisciplinary brainstorming, competi- 
tive analysis, intra-office peer review of production, 
and elicitation of outside expertise. 


Heuer emphasizes both the value and the dangers of mental 
models, or mind-sets. In the book’s opening chapter, entitled “Think- 
ing About Thinking,” he notes that: 


[Analysts] construct their own version of “reality” on the 
basis of information provided by the senses, but this sen- 
sory input is mediated by complex mental processes that 
determine which information is attended to, how it is orga- 
nized, and the meaning attributed to it. What people per- 
ceive, how readily they perceive it, and how they process 
this information after receiving it are all strongly influ- 
enced by past experience, education, cultural values, role 
requirements, and organizational norms, as well as by the 
specifics of the information received. 


This process may be visualized as perceiving the world 
through a lens or screen that channels and focuses and 
thereby may distort the images that are seen. To achieve the 
clearest possible image...analysts need more than 
information... They also need to understand the lenses 
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through which this information passes. These lenses are 
known by many terms—mental models, mind-sets, biases, 
or analytic assumptions. 


In essence, Heuer sees reliance on mental models to simplify 
and interpret reality as an unavoidable conceptual mechanism for 
intelligence analysts—often useful, but at times hazardous. What is 
required of analysts, in his view, is a commitment to challenge, refine, 
and challenge again their own working mental models, precisely 
because these steps are central to sound interpretation of complex and 
ambiguous issues. 

Throughout the book, Heuer is critical of the orthodox prescrip- 
tion of “more and better information” to remedy unsatisfactory ana- 
lytic performance. He urges that greater attention be paid instead to 
more intensive exploitation of information already on hand, and that 
in so doing, analysts continuously challenge and revise their mental 
models. 

Heuer sees mirror-imaging as an example of an unavoidable 
cognitive trap. No matter how much expertise an analyst applies to 
interpreting the value systems of foreign entities, when the hard evi- 


dence runs out the tendency to project the analyst’s own mind-set 
takes over. In Chapter 4, Heuer observes: 


To see the options faced by foreign leaders as these leaders 
see them, one must understand their values and assump- 
tions and even their misperceptions and misunderstand- 
ings. Without such insight, interpreting foreign leaders’ 
decisions or forecasting future decisions is often nothing 
more than partially informed speculation. Too frequently, 
foreign behavior appears “irrational” or “not in their own 
best interest.” Such conclusions often indicate analysts 
have projected American values and conceptual frame- 
works onto the foreign leaders and societies, rather than 
understanding the logic of the situation as it appears to 
them. 


Competing Hypotheses 

To offset the risks accompanying analysts’ inevitable recourse to 
mirror-imaging, Heuer suggests looking upon analysts’ calculations 
about foreign beliefs and behavior as hypotheses to be challenged. 
Alternative hypotheses need to be carefully considered—especially 
those that cannot be disproved on the basis of available information. 

Heuer’s concept of “Analysis of Competing Hypotheses” (ACH) 
is among his most important contributions to the development of an 
intelligence analysis methodology. At the core of ACH is the notion of 
competition among a series of plausible hypotheses to see which ones 
survive a gauntlet of testing for compatibility with available informa- 
tion. The surviving hypotheses—those that have not been disproved— 
are subjected to further testing. ACH, Heuer concedes, will not always 
yield the right answer. But it can help analysts overcome the cogni- 
tive limitations discussed in his book. 

Some analysts who use ACH follow Heuer’s full eight-step 
methodology. More often, they employ some elements of ACH— 
especially the use of available information to challenge the hypotheses 
that the analyst favors the most. 


Denial and Deception 

Heuer’s path-breaking work on countering denial and deception 
(D&D) was not included as a separate chapter in this volume. But his 
brief references here are persuasive. 

He notes, for example, that analysts often reject the possibility of 
deception because they see no evidence of it. He then argues that 
rejection is not justified under these circumstances. If deception is 
well planned and properly executed, one should not expect to see evi- 
dence of it readily at hand. Rejecting a plausible but unproven hypoth- 
esis too early tends to bias the subsequent analysis, because one does 
not then look for the evidence that might support it. The possibility of 
deception should not be rejected until it is disproved or, at least, until 
a systematic search for evidence has been made and none has been 
found. 
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Heuer’s Impact 


Heuer’s influence on analytic tradecraft began with his first arti- 
cles. CIA officials who set up training courses in the 1980s as part of 
then-DDI Gates’s quest for improved analysis shaped their lesson 
plans partly on the basis of Heuer’s findings. Among these courses 
were a seminar on intelligence successes and failures and another on 
intelligence analysis. The courses influenced scores of DI analysts, 
many of whom are now in the managerial ranks. The designers and 
teachers of Tradecraft 2000 clearly were also influenced by Heuer, as 
reflected in reading selections, case studies, and class exercises. 

Heuer’s work has remained on reading lists and in lesson plans 
for DI training courses offered to all new analysts, as well as courses 
on warning analysis and on countering denial and deception. Senior 
analysts and managers who have been directly exposed to Heuer’s 
thinking through his articles, or through training courses, continue to 
pass his insights on to newer analysts. 


Recommendations 


Heuer’s advice to Agency leaders, managers, and analysts is 
pointed: To ensure sustained improvement in assessing complex 
issues, analysis must be treated as more than a substantive and organi- 
zational process. Attention also must be paid to techniques and tools 
for coping with the inherent limitations on analysts’ mental machin- 
ery. He urges that Agency leaders take steps to: 


¢ Establish an organizational environment that promotes and 
rewards the kind of critical thinking he advocates—for exam- 
ple, analysis on difficult issues that considers in depth a series 
of plausible hypotheses rather than allowing the first credible 
hypothesis to suffice. 


¢ Expand funding for research on the role such mental processes 
play in shaping analytical judgments. An Agency that relies on 
sharp cognitive performance by its analysts must stay abreast of 
studies on how the mind works—i.e., on How analysts reach 
judgments. 


¢ Foster development of tools to assist analysts in assessing infor- 
mation. On tough issues, they need help in improving their 
mental models and in deriving incisive findings from informa- 
tion they already have; they need such help at least as much as 
they need more information. 


1 offer some concluding observations and recommendations, 
rooted in Heuer’s findings and taking into account the tough tradeoffs 
facing intelligence professionals: 


* Commit to a uniform set of tradecraft standards based on the 
insights in this book. Leaders need to know if analysts have 
done their cognitive homework before taking corporate respon- 
sibility for their judgments. Although every analytical issue can 
be seen as one of a kind, I suspect that nearly all such topics fit 
into about a dozen recurring patterns of challenge based largely 
on variations in substantive uncertainty and policy sensitivity. 
Corporate standards need to be established for each such cate- 
gory. And the burden should be put on managers to explain why 
a given analytical assignment requires deviation from the stan- 
dards. | am convinced that if tradecraft standards are made uni- 
form and transparent, the time saved by curtailing personalistic 
review of quick-turnaround analysis (e.g., “It reads better to me 
this way”) could be “re-invested” in doing battle more effec- 
tively against cognitive pitfalls. (“Regarding point 3, let’s talk 
about your assumptions.”) 


¢ Pay more honor to “doubt.” Intelligence leaders and policy- 
makers should, in recognition of the cognitive impediments to 
sound analysis, establish ground rules that enable analysts, after 
doing their best to clarify an issue, to express doubts more 
openly. They should be encouraged to list gaps in information 
and other obstacles to confident judgment. Such conclusions as 
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“We do not know” or “There are several potentially valid ways 
to assess this issue” should be regarded as badges of sound 
analysis, not as dereliction of analytic duty. 


¢ Find a couple of successors to Dick Heuer. Fund their 
research. Heed their findings. 
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PART OQNE—OUR MENTAL MACHINERY 
Chapter 1 
Thinking About Thinking 


Of the diverse problems that impede accurate intelligence analy- 
sis, those inherent in human mental processes are surely among the 
most important and most difficult to deal with. Intelligence analysis is 
fundamentally a mental process, but understanding this process is 
hindered by the lack of conscious awareness of the workings of our 
own minds. 

A basic finding of cognitive psychology is that people have no 
conscious experience of most of what happens in the human mind. 
Many functions associated with perception, memory, and information 
processing are conducted prior to and independently of any con- 
scious direction. What appears spontaneously in consciousness is the 
result of thinking, not the process of thinking. 

Weaknesses and biases inherent in human thinking processes 
can be demonstrated through carefully designed experiments. They 
can be alleviated by conscious application of tools and techniques 
that should be in the analytical tradecraft toolkit of all intelligence 
analysts. 
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“When we speak of improving the mind we are usually refer- 
ring to the acquisition of information or knowledge, or to the type of 
thoughts one should have, and not to the actual functioning of the 
mind. We spend little time monitoring our own thinking and compar- 
ing it with a more sophisticated ideal.” '' 


James L. Adams, Conceptual Blockbusting: A Guide to Better Ideas (New York: W.W. Norton, 
second edition, 1980), p. 3. 


When we speak of improving intelligence analysis, we are usu- 
ally referring to the quality of writing, types of analytical products, 
relations between intelligence analysts and intelligence consumers, or 
organization of the analytical process. Little attention is devoted to 
improving how analysts think. 

Thinking analytically is a skill like carpentry or driving a car. It 
can be taught, it can be learned, and it can improve with practice. But 
like many other skills, such as riding a bike, it is not learned by sitting 
in a classroom and being told how to do it. Analysts learn by doing. 
Most people achieve at least a minimally acceptable level of analytical 
performance with little conscious effort beyond completing their edu- 
cation. With much effort and hard work, however, analysts can 
achieve a level of excellence beyond what comes naturally. 

Regular running enhances endurance but does not improve tech- 
nique without expert guidance. Similarly, expert guidance may be 
required to modify long-established analytical habits to achieve an 
optimal level of analytical excellence. An analytical coaching staff to 
help young analysts hone their analytical tradecraft would be a valu- 
able supplement to classroom instruction. 

One key to successful learning is motivation. Some of CIA’s best 
analysts developed their skills as a consequence of experiencing ana- 
lytical failure early in their careers. Failure motivated them to be more 
self-conscious about how they do analysis and to sharpen their think- 
ing process. 

This book aims to help intelligence analysts achieve a higher 
level of performance. It shows how people make judgments based on 
incomplete and ambiguous information, and it offers simple tools and 
concepts for improving analytical skills. 

Part I identifies some limitations inherent in human mental pro- 
cesses. Part II discusses analytical tradecraft—simple tools and 
approaches for overcoming the: limitations and thinking more sys- 
tematically. Chapter 8, “Analysis of Competing Hypotheses,” is argu- 
ably the most important single chapter. Part II] presents information 
about cognitive biases—the technical term for predictable mental 
errors caused by simplified information processing strategies. A final 


chapter presents a checklist for analysts and recommendations for 
how managers of intelligence analysis can help create an environment 
in which analytical excellence flourishes. 

Herbert Simon first advanced the concept of “bounded” or lim- 
ited rationality." Because of limits in human mental capacity, he 
argued, the mind cannot cope directly with the complexity of the 
world. Rather, we construct a simplified mental model of reality and 
then work with this model. We behave rationally within the confines 
of our mental model, but this model is not always well adapted to the 
requirements of the real world. The concept of bounded rationality has 
come to be recognized widely, though not universally, both as an 
accurate portrayal of human judgment and choice and as a sensible 
adjustment to the limitations inherent in how the human mind func- 
tions."* 

Much psychological research on perception, memory, attention 
span, and reasoning capacity documents the limitations in our “men- 
tal machinery” identified by Simon. Many scholars have applied these 
psychological insights to the study of international political behavior. 
A similar psychological perspective underlies some writings on intel- 
ligence failure and strategic surprise." 


* Herbert Simon, Models of Man, 1957. 

James G. March., “Bounded Rationality, Ambiguity, and the Engineering of Choice.” in David 
E. Bell, Howard Raiffa, and Amos Tversky, eds., Decision Making: Descriptive, Normative, and 
Prescriptive Interactions (Cambridge University Press, 1988). 

* Among the early scholars who wrote on this subject were Joseph De Rivera, The Psychological 
Dimension of Foreign Policy (Columbus, OH: Merrill, 1968), Alexander George and Richard 
Smoke, Deterrence in American Foreign Policy (New York: Columbia University Press, 1974), and 
Robert Jervis, Perception and Misperception in International Politics (Princeton, NJ: Princeton 
University Press. 1976). 

* Christopher Brady, “Intelligence Failures: Plus Ca Change ...“ Intelligence and Naticnal Secu- 
rity, Vol. 8, No. 4 (October 1993). N. Cigar, “Iraq's Strategic Mindset and the Gulf War: Blueprint 
for Defeat,” The Journal of Strategic Studies, Vol. 15, No. | (March 1992). J. J. Wirtz, The Tet 
Offensive. Intelligence Failure in War (New York, 1991). Ephraim Kam, Surprise Attack (Harvard 
University Press, 1988). Richard Betis, Surprise Atack: Lessons for Defense Planning (Brook- 
ings, 1982). Abraham Ben-Zvi, “The Study of Surprise Attacks,” British Journal of International 
Studies, Vol. 5 (1979). Iran: Evaluation of Intelligence Performance Prior to November 1978 
(Staff Report, Subcommittee on Evaluation, Permanent Select Committee on Intelligence, US 
House of Representatives, January 1979). Richard Betts, “Analysis, War and Decision: Why Intel- 
ligence Failures Are Inevitable,” World Politics, Vol. 31, No. 1 (October 1978). Richard W. Shry- 
ock, “The Intelligence Community Post-Mortem Program, 1973-1975.” Studies in Intelligence, 
Vol. 21, No. | (Fall 1977). Avi Schlaim, “Failures in National Intelligence Estimates: The Case of 
the Yom Kippur War.” World Politics, Vol. 28 (April 1976). Michael Handel, Perception, Decep- 
tion, and Surprise: The Case of the Yom Kippur War Jerusalem: Leonard Davis Institute of Inter- 
national Relations, Jerusalem Paper No. 19, 1976). Klaus Knorr, “Failures in National Intelligence 
Estimates: The Case of the Cuban Missiles.” World Politics, Vol. 16 (1964). 


This book differs from those works in two respects. It analyzes 
problems from the perspective of intelligence analysts rather than pol- 
icymakers. And it documents the impact of mental processes largely 
through experiments in cognitive psychology rather than through 
examples from diplomatic and military history. 

A central focus of this book is to illuminate the role of the 
observer in determining what is observed and how it is interpreted. 
People construct their own version of “reality” on the basis of infor- 
mation provided by the senses, but this sensory input is mediated by 
complex mental processes that determine which information is 
attended to, how it is organized, and the meaning attributed to it. What 
people perceive, how readily they perceive it, and how they process 
this information after receiving it are all strongly influenced by past 
experience, education, cultural values, role requirements, and organi- 
zational norms, as well as by the specifics of the information received. 

This process may be visualized as perceiving the world through 
a lens or screen that channels and focuses and thereby may distort the 
images that are seen. To achieve the clearest possible image of China, 
for example, analysts need more than information on China. They also 
need to understand their own lenses through which this information 
passes. These lenses are known by many terms—mental models, 
mind-sets, biases, or analytical assumptions. 

In this book, the terms mental model and mind-set are used more 
or less interchangeably, although a mental model is likely to be better 
developed and articulated than a mind-set. An analytical assumption is 
one part of a mental model or mind-set. The biases discussed in this 
book result from how the mind works and are independent of any sub- 
stantive mental model or mind-set. 

Before obtaining a license to practice, psychoanalysts are 
required to undergo psychoanalysis themselves in order to become 
more aware of how their own personality interacts with and conditions 
their observations of others. The practice of psychoanalysis has not 
been so successful that its procedures should be emulated by the intel- 
ligence and foreign policy community. But the analogy highlights an 
interesting point: Intelligence analysts must understand themselves 
before they can understand others. Training is needed to (a) increase 
self-awareness concerning generic problems in how people perceive 
and make analytical judgmeo's concerning foreign events, and (b) 
provide guidance and practice in overcoming these problems. 


Not enough training is focused in this direction—that is, inward 
toward the analyst's own thought processes. Training of intelligence 
analysts generally means instruction in organizational procedures, 
methodological techniques, or substantive topics. More training time 
should be devoted to the mental act of thinking or analyzing. It is sim- 
ply assumed, incorrectly, that analysts know how to analyze. This 
book is intended to support training that examines the thinking and 
reasoning processes involved in intelligence analysis. 

As discussed in the next chapter, mind-sets and mental models 
are inescapable. They are, in essence, a distillation of all that we think 
we know about a subject. The problem is how to ensure that the mind 
remains open to alternative interpretations in a rapidly changing 
world. 

The disadvantage of a mind-set is that it can color and control 
our perception to the extent that an experienced specialist may be 
among the last to see what is really happening when events take a new 
and unexpected turn. When faced with a major paradigm shift, ana- 
lysts who know the most about a subject have the most to unlearn. 
This seems to have happened before the reunification of Germany, for 
example. Some German specialists had to be prodded by their more 
generalist supervisors to accept the significance of the dramatic 
changes in progress toward reunification of East and West Germany. 

The advantage of mind-sets is that they help analysts get the pro- 
duction out on time and keep things going effectively between those 
watershed events that become chapter headings in the history books.'*° 

A generation ago, few intelligence analysts were self-conscious 
and introspective about the process by which they did analysis. The 
accepted wisdom was the “common sense” theory of knowledge—that 
to perceive events accurately it was necessary only to open one’s eyes, 
look at the facts, and purge oneself of all preconceptions and preju- 
dices in order to make an objective judgment. 

Today, there is greatly increased understanding that intelligence 
analysts do not approach their tasks with empty minds. They start 
with a set of assumptions about how events normally transpire in the 
area for which they are responsible. Although this changed view is 
becoming conventional wisdom, the Intelligence Community has only 
begun to scratch the surface of its implications. 


* This wording is from a discussion with veteran CIA analyst, author, and teacher Jack Davis. 


If analysts’ understanding of events is greatly influenced by the 
mind-set or mental model through which they perceive those events, 
should there not be more research to explore and document the impact 
of different mental models?" 

The reaction of the Intelligence Community to many problems is 
to collect more information, even though analysts in many cases 
already have more information than they can digest. What analysts 
need is more truly useful information—mostly reliable HUMINT 
from knowledgeable insiders—to help them make good decisions. Or 
they need a more accurate mental model and better analytical tools to 
help them sort through, make sense of, and get the most out of the 
available ambiguous and conflicting information. 

Psychological research also offers to intelligence analysts addi- 
tional insights that are beyond the scope of this book. Problems are 
not limited to how analysts perceive and process information. Intelli- 
gence analysts often work in small groups and always within the con- 
text of a large, bureaucratic organization. Problems are inherent in the 
processes that occur at all three levels—individual, small group, and 
organization. This book focuses on problems inherent in analysts’ 
mental processes, inasmuch as these are probably the most insidious. 
Analysts can observe and get a feel for these problems in small-group 
and organizational processes, but it is very difficult, at best, to be self- 
conscious about the workings of one’s own mind. 


Graham Allison’s work on the Cuban missile crisis (Essence of Decision, Little, Brown & Co., 
1971) is an example of what I have in mind. Allison identified three alternative assumptions about 
how governments work—a rational actor model, an organizational process model, and a bureau- 
cratic politics model. He then showed how an analyst's implicit assumptions about the most appro- 
priate model for analyzing a foreign government's behavior can cause him or her to focus on 
different evidence and arrive at different conclusions. Another example is my own analysis of five 
alternative paths for making counterintelligence judgments in the controversial case of KGB defec- 
tor Yuriy Nosenko: Richards J. Heuer, Jr., “Nosenko: Five Paths to Judgme:t,” Studies in Intelli- 
gence, Vol. 31, No. 3 (Fall 1987), originally classified Secret but declassified and published in H. 
Bradford Westerfield, ed., /nside CIA's Private World: Declassified Articles from the Agency's 
Internal Journal 1955-1992 (New Haven: Yale University Press, 1995). 


Chapter 2 


Perception: Why Can’t We See 
What Is There To Be Seen? 


The process of perception links people to their environment and 
is critical to accurate understanding of the world about us. Accurate 
intelligence analysis obviously requires accurate perception. Yet 
research into human perception demonstrates that the process is beset 
by many pitfalls. Moreover, the circumstances under which intelli- 
gence analysis is conducted are precisely the circumstances in which 
accurate perception tends to be most difficult. This chapter discusses 
perception in general, then applies this information to illuminate 
some of the difficulties of intelligence analysis."* 
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People tend to think of perception as a passive process. We see, 
hear, smell, taste or feel stimuli that impinge upon our senses. We 
think that if we are at all objective, we record what is actually there. 
Yet perception is demonstrably an active rather than a passive process; 
it constructs rather than records “reality.” Perception implies under- 
Standing as well as awareness. It is a process of inference in which 
people construct their own version of reality on the basis of informa- 
tion provided through the five senses. 

As already noted, what people in general and analysts in particu- 
lar perceive, and how readily they perceive it, are strongly influenced 
by their past experience, education, cultural values, and role require- 
ments, as well as by the stimuli recorded by their receptor organs. 

Many experiments have been conducted to show the extraordi- 
nary extent to which the information obtained by an observer depends 


‘* An earlier version of this article was published as part of “Cognitive Factors in Deception and 
Counterdeception,” in Donald C. Daniel and Katherine L. Herbig, eds., Strategic Military Decep- 
tion (Pergamon Press, 1982). 


upon the observer’s own assumptions and preconceptions. For exam- 
ple, when you looked at Figure | above, what did you see? Now refer 
to the footnote for a description of what is actually there.’ Did you 
perceive Figure | correctly? If so, you have exceptional powers of 
observation, were lucky, or have seen the figure before. This simple 
experiment demonstrates one of the most fundamental principles con- 
cerning perception: 


We tend to perceive what we expect to perceive. 


A corollary of this principle is that it takes more information, 
and more unambiguous information, to recognize an unexpected phe- 
nomenon than an expected one. 

One classic experiment to demonstrate the influence of expecta- 
tions on perception used playing cards, some of which were gim- 
micked so the spades were red and the hearts black. Pictures of the 
cards were flashed briefly on a screen and, needless to say, the test 
subjects identified the normal cards more quickly and accurately than 
the anomalous ones. After test subjects became aware of the exist- 
ence of red spades and black hearts, their performance with the gim- 
micked cards improved but still did not approach the speed or 
accuracy with which normal cards could be identified.” 


The article is written twice in each of the three phrases. This is commonly overlooked because 
perception is influenced by our expectations about how these familiar phrases are normally written. 
* Jerome S. Bruner and Leo Postman, “On the Perception of Incongruity: A Paradigm,” in Jerome 
S. Bruner and David Kraut, eds., Perception and Personality: A Symposium (New York: Green- 
wood Press, 1968). 


This experiment shows that patterns of expectation become so 
deeply embedded that they continue to influence perceptions even 
when people are alerted to and try to take account of the existence of 
data that do not fit their preconceptions. Trying to be objective does 
not ensure accurate perception. 

The position of the test subject identifying playing cards is anal- 
ogous to that of the intelligence analyst or government leader trying to 
make sense of the paper flow that crosses his or her desk. What is 
actually perceived in that paper flow, as well as how it is interpreted, 
depends in part, at least, on the analyst's patterns of expectation. Ana- 
lysts do not just have expectations about the color of hearts and 
spades. They have a set of assumptions and expectations about the 
motivations of people and the processes of government in foreign 
countries. Events consistent with these expectations are perceived and 
processed easily, while events that contradict prevailing expectations 
tend to be ignored or distorted in perception. Of course, this distor- 
tion is a subconscious or pre-conscious process, as illustrated by how 
you presumably ignored the extra words in the triangles in Figure 1. 

This tendency of people to perceive what they expect to per- 
ceive is more important than any tendency to perceive what they want 
to perceive. In fact, there may be no real tendency toward wishful 
thinking. The commonly cited evidence supporting the claim that peo- 
ple tend to perceive what they want to perceive can generally be 
explained equally well by the expectancy thesis.*' 

Expectations have many diverse sources, including past experi- 
ence, professional training, and cultural and organizational norms. All 
these influences predispose analysts to pay particular attention to cer- 
tain kinds of information and to organize and interpret this informa- 
tion in certain ways. Perception is also influenced by the context in 
which it occurs. Different circumstances evoke different sets of expec- 
tations. People are more attuned to hearing footsteps behind them 
when walking in an alley at night than along a city street in daytime, 
and the meaning attributed to the sound of footsteps will vary under 
these differing circumstances. A military intelligence analyst may be 
similarly tuned to perceive indicators of potential conflict. 


For discussion of the ambiguous evidence concerning the impact of desires and fears on judg- 
ment, see Robert Jervis, Perception and Misperception in International Politics (Princeton, NJ: 
Princeton University Press, 1976), Chapter 10. 


Patterns of expectations tell analysts, subconsciously, what to 
look for, what is important, and how to interpret what is seen. These 
patterns form a mind-set that predisposes analysts to think in certain 
ways. A mind-set is akin to a screen or lens through which one per- 
ceives the world. 

There is a tendency to think of a mind-set as something bad, to 
be avoided. According to this line of argument, one should have an 
open mind and be influenced only by the facts rather than by precon- 
ceived notions! That is an unreachable ideal. There is no such thing as 
“the facts of the case.” There is only a very selective subset of the 
overall mass of data to which one has been subjected that one takes as 
facts and judges to be relevant to the question at issue. 

Actually, mind-sets are neither good nor bad; they are unavoid- 
able. People have no conceivable way of coping with the volume of 
stimuli that impinge upon their senses, or with the volume and com- 
plexity of the data they have to analyze, without some kind of simpli- 
fying preconceptions about what to expect, what is important, and 
what is related to what. “There is a grain of truth in the otherwise per- 
nicious maxim that an open mind is an empty mind.” Analysts do 
not achieve objective analysis by avoiding preconceptions; that would 


be ignorance or self-delusion. Objectivity is achieved by making basic 
assumptions and reasoning as explicit as possible so that they can be 
challenged by others and analysts can, themselves, examine their 
validity. 

One of the most important characteristics of mind-sets is: 


Mind-sets tend to be quick to form but resistant to change. 


Figure 2 illustrates this principle by showing part of a longer 
series of progressively modified drawings that change almost imper- 
ceptibly from a man into a woman.*’ The right-hand drawing in the 
top row, when viewed alone, has equal chances of being perceived as 
a man or a woman. When test subjects are shown the entire series of 
drawings one by one, their perception of this intermediate drawing is 
biased according to which end of the series they started from. Test 
subjects who start by viewing a picture that is clearly a man are biased 
in favor of continuing to see a man long after an “objective observer” 
* Richard Betts, “Analysis, War and Decision: Why Intelligence Failures are Inevitable”, World 


Politics, Vol. XXX1 (October 1978), p. 84. 
‘ Drawings devised by Gerald Fisher in 1967. 
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Impressions resist change. 


(for example, an observer who has seen only a single picture) recog- 
nizes that the man is now a woman. Similarly, test subjects who start 
at the woman end of the series are biased in favor of continuing to see 
a woman. Once an observer has formed an image—that is, once he or 
she has developed a mind-set or expectation concerning the phenome- 
non being observed—this conditions future perceptions of that phe- 
nomenon. 
This is the basis for another general principle of perception: 


New information is assimilated to existing images 


This principle explains why gradual, evolutionary change often 
goes unnoticed. It also explains the phenomenon that an intelligence 
analyst assigned to work on a topic or country for the first time may 
generate accurate insights that have been overlooked by experienced 
analysts who have worked on the same problem for 10 years. A fresh 
perspective is sometimes useful; past experience can handicap as well 
as aid analysis. This tendency to assimilate new data into pre-existing 


It is difficult to look at the same information from different 
perspectives. 


images is greater “the more ambiguous the information, the more con- 
fident the actor is of the validity of his image, and the greater his com- 


924 


mitment to the established view. 

The drawing in Figure 3 provides the reader an opportunity to 
test for him or herself the persistence of established images.** Look at 
Figure 3. What do you see—an old woman or a young woman? Now 
look again to see if you can visually and mentally reorganize the data 
to form a different image—that of a young woman if your original 


* Jervis, p. 195. 
* This picture was originally published in Puck magazine in 1915 as a cartoon entitled “My Wife 
and My Mother-in-Law.” 


perception was of an old woman, or of the old woman if you first per- 
ceived the young one. If necessary, look at the footnote for clues to 
help you identify the other image.” Again, this exercise illustrates the 
principle that mind-sets are quick to form but resistant to change. 

When you have seen Figure 3 from both perspectives, try shift- 
ing back and forth from one perspective to the other. Do you notice 
some initial difficulty in making this switch? One of the more diffi- 
cult mental feats is to take a familiar body of data and reorganize it 
visually or mentally to perceive it from a different perspective. Yet 
this is whai intelligence analysts are constantly required to do. In 
order to understand international interactions, analysts must under- 
stand the situation as it appears to each of the opposing forces, and 
constantly shift back and forth from one perspective to the other as 
they try to fathom how each side interprets an ongoing series of inter- 
actions. Trying to perceive an adversary’s interpretations of interna- 
tional events, as well as US interpretations of those same events, is 
comparable to seeing both the old and young woman in Figure 3. 
Once events have been perceived one way, there is a natural resis- 
tance to other perspectives. 

A related point concerns the impact of substandard conditions of 
perception. The basic principle is: 


Initial exposure to blurred or ambiguous stimuli interferes 
with accurate perception even after more and better informa- 
tion becomes available. 


This effect has been demonstrated experimentally by projecting 
onto a screen pictures of common, everyday subjects such as a dog 
standing on grass, a fire hydrant, and an aerial view of a highway clo- 
verleaf intersection.*’ The initial projection was blurred in varying 
degrees, and the pictures were then brought into focus slowly to deter- 
mine at what point test subjects could identify them correctly. 

This experiment showed two things. First, those who started 
viewing the pictures when they were most out of focus had more diffi- 
culty identifying them when they became clearer than those who 


* The old woman's nose, mouth, and eye are, respectively, the young woman's chin, necklace, and 
ear. The old woman is seen in profile looking left. The young woman is also looking left, but we 
see her mainly from behind so most facial features are not visible. Her eyelash, nose, and the curve 
of her cheek may be seen just above the old woman's nose. 

Jerome S. Bruner and Mary C. Potter, “Interference in Visual Recognition,” Science, Vol. 144 
(1964), pp. 424-25. 


Started viewing at a less blurred stage. In other words, the greater the 
initial blur, the clearer the picture had to be before people could rec- 
ognize it. Second, the longer people were exposed to a blurred picture, 
the clearer the picture had to be before they could recognize it. 

What happened in this experiment is what presumably happens 
in real life; despite ambiguous stimuli, people form some sort of tenta- 
tive hypothesis about what they see. The longer they are exposed to 
this blurred image, the greater confidence they develop in this initial 
and perhaps erroneous impression, so the greater the impact this ini- 
tial impression has on subsequent perceptions. For a time, as the pic- 
ture becomes clearer, there is no obvious contradiction; the new data 
are assimilated into the previous image, and the initial interpretation is 
maintained until the contradiction becomes so obvious that it forces 
itself upon our consciousness. 

The early but incorrect impression tends to persist because the 
amount of information necessary to invalidate a hypothesis is consid- 
erably greater than the amount of information required to make an ini- 
tial interpretation. The problem is not that there is any inherent 
difficulty in grasping new perceptions or new ideas, but that estab- 
lished perceptions are so difficult to change. People form impressions 
on the basis of very little information, but once formed, they do not 
reject or change them unless they obtain rather solid evidence. Ana- 
lysts might seek to limit the adverse impact of this tendency by sus- 
pending judgment for as long as possible as new information is being 
received. 


Implications for Intelligence Analysis 


Comprehending the nature of perception has significant implica- 
tions for understanding the nature and limitations of intelligence anal- 
ysis. The circumstances under which accurate perception is most 
difficult are exactly the circumstances under which intelligence analy- 
sis is generally conducted—dealing with highly ambiguous situations 
on the basis of information that is processed incrementally under pres- 
sure for early judgment. This is a recipe for inaccurate perception. 

Intelligence seeks to illuminate the unknown. Almost by defini- 
tion, intelligence analysis deals with highly ambiguous situations. As 
previously noted, the greater the ambiguity of the stimult, the greater 
the impact of expectations and pre-existing images on the perception 
of that stimuli. Thus, despite maximum striving for objectivity, the 


intelligence analyst's own preconceptions are likely to exert a greater 
impact on the analytical product than in other fields where an analyst 
is working with less ambiguous and less discordant information. 

Moreover, the intelligence analyst is among the first to look at 
new problems at an early stage when the evidence is very fuzzy 
indeed. The analyst then follows a problem as additional increments 
of evidence are received and the picture gradually clarifies—as hap- 
pened with test subjects in the experiment demonstrating that initial 
exposure to blurred stimuli interferes with accurate perception even 
after more and better information becomes available. If the results of 
this experiment can be generalized to apply to intelligence analysts, 
the experiment suggests that an analyst who starts observing a poten- 
tial problem situation at an early and unclear stage is at a disadvantage 
as compared with others, such as policymakers, whose first exposure 
may come at a later stage when more and better information is avail- 
able. 

The receipt of information in small increments over time also 
facilitates assimilation of this information into the analyst's existing 
views. No one item of information may be sufficient to prompt the 
analyst to change a previous view. The cumulative message inherent 
in many pieces of information may be significant but is attenuated 
when this information is not examined as a whole. The Intelligence 
Community's review of its performance before the 1973 Arab-Israeli 
War noted: 


The problem of incremental analysis—especially as it applies to 
the current intelligence process—was also at work in the period 
preceding hostilities. Analysts, according to their own accounts, 
were often proceeding on the basis of the day's take, hastily com- 
paring it with materia! received the previous day. They then pro- 
duced in ‘assembly line fashion’ items which may have reflected 
perceptive intuition but which [did not] accrue from a systematic 
consideration of an accumulated body of integrated evidence.” 


And finally, the intelligence analyst operates in an environment 
that exerts strong pressures for what psychologists call premature clo- 
sure. Customer demand for interpretive analysis is greatest within two 


* The Performance of the Intelligence Community Before the Arab-Asrach War of October 1973) A 
Preluomnary Post-Mortem Report, December 1973. The one-paragraph excerpt trom this post-mor- 
tem, as quoted in the text above, has been approved for public release. as was the title of the post. 
mortem, although that document as a whole remains classifed 


or three days after an event occurs. The system requires the intelli- 
gence analyst to come up with an almost instant diagnosis before suf- 
ficient hard information, and the broader background information that 
may be needed to gain perspective, become available to make possible 
a well-grounded judgment. This diagnosis can only be based upon the 
analyst's preconceptions concerning how and why events normally 
transpire in a given society. 

As time passes and more information is received, a fresh look at 
all the evidence might suggest a different explanation. Yet, the percep- 
tion experiments indicate that an early judgment adversely affects the 
formation of {uture perceptions. Once an observer thinks he or she 
knows what is happening, this perception tends to resist change. New 
data received incrementally can be fit easily into an analyst's previous 
image. This perceptual bias is reinforced by organizational pressures 
favoring consistent interpretation; once the analyst is committed in 
writing, both the analyst and the organization have a vested interest in 
maintaining the original assessment. 

That intelligence analysts perform as well as they do is testi- 
mony to their generally sound judgment, training, and dedication in 
performing a dauntingly difficult task. 

The problems outlined here have implications for the manage- 
ment as well as the conduct of analysis. Given the difficulties inher- 
ent in the human processing of complex information, a prudent 
management system should: 


¢ Encourage products that clearly delineate their assumptions and 
chains of inference and that specify the degree and source of 
uncertainty involved in the conclusions. 


¢ Support analyses that periodically re-examine key problems 
from the ground up in order to avoid the pitfalls of the incre- 
mental approach. 


* Emphasize procedures that expose and elaborate alternative 
points of view. 


* Educate consumers about the limitations as well as the capabil- 
ities of intelligence analysis; define a set of realistic expecta- 
tions as a standard against which to judge analytical 
performance. 


Chapter 3 


Memory: How Do We Remember 
What We Know? 


Differences between stronger and weaker analytical perfor- 
mance are attributable in large measure to differences in the organi- 
zation of data and experience in analysts’ long-term memory. The 
contents of memory form a continuous input into the analytical pro- 
cess, and anything that influences what information is remembered or 
retrieved from memory also influences the outcome of analysis. 

This chapter discusses the capabilities and limitations of sev- 
eral components of the memory system. Sensory information storage 
and short-term memory are beset by severe limitations of capacity, 
while long-term memory, for all practical purposes, has a virtually 
infinite capacity. With long-term memory, the problems concern get- 
ting information into it and retrieving information once it is there, not 
physical limits on the amount of information that may be stored. 
Understanding how memory works provides insight into several ana- 
lytical strengths and weaknesses. 
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Components of the Memory System 


What is commonly called memory is not a single, simple func- 
tion. It is an extraordinarily complex system of diverse componenrits 
and processes. There are at least three, and very likely more, distinct 
memory processes. The most important from the standpoint of this 
discussion and best documented by scientific research are sensory 
information storage (SIS), short-term memory (STM), and long-term 


memory (LTM).” Each differs with respect to function, the form of 
information held, the length of time information is retained, and the 
amount of information-handling capacity. Memory researchers also 
posit the existence of an interpretive mechanism and an overall mem- 
ory monitor or control mechanism that guides interaction among vari- 
ous elements of the memory system. 


Sensory Information Storage 

Sensory information storage holds sensory images for several 
tenths of a second after they are received by the sensory organs. The 
functioning of SIS may be observed if you close your eyes, then open 
and close them again as rapidly as possible. As your eyes close, notice 
how the visual image is maintained for a fraction of a second before 
fading. Sensory information storage explains why a movie film shot at 
16 separate frames per second appears as continuous movement rather 
than a series of still pictures. A visual trace is generally retained in 
SIS for about one-quarter of a second. It is not possible to consciously 
extend the time that sensory information is held in SIS. The function 
of SIS is to make it possible for the brain to work on processing a sen- 
sory event for longer than the duration of the event itself 


Short-Term Memory 

Information passes from SIS into short-term memory, where 
again it is held for only a short period of time—a few seconds or min- 
utes. Whereas SIS holds the complete image, STM stores only the 
interpretation of the image. If a sentence is spoken, SIS retains the 
sounds, while STM holds the words formed by these sounds. 

Like SIS, short-term memory holds information temporarily, 
pending further processing. This processing includes judgments con- 
cerning meaning, relevance, and significance, as well as the mental 
actions necessary to integrate selected portions of the information into 
long-term memory. When a person forgets immediately the name of 
someone to whom he or she has just been introduced, it is because the 
name was not transferred from short-term to long-term memory. 


* Memory researchers do not employ uniform terminology. Sensory information storage is also 
known as sensory register, sensory store, and eidetic and echoic memory. Short- and long-term 
memory are also referred to as primary and secondary memory. A variety of other terms are in use 
as well. | have adopted the terminology used by Peter H. Lindsay and Donald A. Norman in their 
text on Human Information Processing (New York: Academic Press, 1977). This entire chapter 
draws heavily from Chapters 8 through || of the Lindsay and Norman book. 


A central characteristic of STM is the severe limitation on its 
capacity. A person who is asked to listen to and repeat a series of 10 
or 20 names or numbers normally retains only five or six items. Com- 
monly it is the last five or six. If one focuses instead on the first items, 
STM becomes saturated by this effort, and the person cannot concen- 
trate on and recall the last items. People make a choice where to focus 
their attention. They can concentrate on remembering or interpreting 
or taking notes on information received moments ago, or pay attention 
to information currently being received. Limitations on the capacity of 
short-term memory often preclude doing both. 

Retrieval of information from STM is direct and immediate 
because the information has never left the conscious mind. Informa- 
tion can be maintained in STM indefinitely by a process of 
“rehearsal”—repeating it over and over again. But while rehearsing 
some items to retain them in STM, people cannot simultaneously add 
new items. The severe limitation on the amount of information retain- 
able in STM at any one time is physiological, and there is no way to 
overcome it. This is an important point that will be discussed below in 
connection with working memory and the utility of external memory 
aids. 


Long-Term Memory 

Some information retained in STM is processed into long-term 
memory. This information on past experiences is filed away in the 
recesses of the mind and must be retrieved before it can be used. In 
contrast to the immediate recall of current experience from STM, 
retrieval of information from LTM is indirect and sometimes labori- 
ous. 

Loss of detail as sensory stimuli are interpreted and passed from 
SIS into STM and then into LTM is the basis for the phenomenon of 
selective perception discussed in the previous chapter. It imposes lim- 
its On subsequent stages of analysis, inasmuch as the lost data can 
never be retrieved. People can never take their mind back to what was 
actually there in sensory information storage or short-term memory. 
They can only retrieve their interpretation of what they thought was 
there as stored in LTM. 

There are no practical limits to the amount of information that 
may be stored in LTM. The limitations of LTM are the difficulty of 
processing information into it and retrieving information from it. 
These subjects are discussed below. 
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The three memory processes comprise the storehouse of infor- 
mation or database that we call memory, but the total memory system 
must include other features as well. Some mental process must deter- 
mine what information is passed from SIS into STM and from STM 
into LTM; decide how to search the LTM data base and judge whether 
further memory search is likely to be productive; assess the relevance 
of retrieved information; and evaluate potentially contradictory data. 

To explain the operation of the total memory system, psycholo- 
gists posit the existence of an interpretive mechanism that operates on 
the data base and a monitor or central control mechanism that guides 
and oversees the operation of the whole system. Little is known of 
these mechanisms and how they relate to other mental processes. 

Despite much research on memory, little agreement exists on 
many critical points. What is presented here is probably the lowest 
common denominator on which most researchers would agree. 

Organization of Information in Long-Term Memory. Physi- 
cally, the brain consists of roughly 10 billion neurons, each analogous 
to a computer chip capable of storing information. Each neuron has 
octopus-like arms called axons and dendrites. Electrical impulses flow 
through these arms and are ferried by neurotransmitting chemicals 
across what is called the synaptic gap between neurons. Memories are 
stored as patterns of connections between neurons. When two neu- 
rons are activated, the connections or “synapses” between them are 
strengthened. 

As you read this chapter, the experience actually causes physi- 
cal changes in your brain. “In a matter of seconds, new circuits are 
formed that can change forever the way you think about the world.” 

Memory records a lifetime of experience and thoughts. Such a 
massive data retrieval mechanism, like a library or computer system, 
must have an organizational structure; otherwise information that 
enters the system could never be retrieved. Imagine the Library of 
Congress if there were no indexing system. 

There has been considerable research on how information is 
organized and represented in memory, but the findings remain specu- 
lative. Current research focuses on which sections of the brain pro- 
cess various types of information. This is determined by testing 


“ George Johnson, In the Palaces of Memory: How We Build the Worlds Inside Our Heads. Vintage 
Books, 1992, p. xi. 
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patients who have suffered brain damage from strokes and trauma or 
by using functional magnetic resonance imaging (fMRI) that “lights 
up” the active portion of the brain as a person speaks, reads, writes, or 
listens. 

None of the current theories seems to encompass the full range 
or complexity of memory processes, which include memory for sights 
and sounds, for feelings, and for belief systems that integrate informa- 
tion on a large number of concepts. However useful the research has 
been for other purposes, analysts’ needs are best served by a very sim- 
ple image of the structure of memory. 

imagine memory as a massive, multidimensional spider web. 
This image captures what is, for the purposes of this book, perhaps 
the most important property of information stored in memory—its 
interconnectedness. One thought leads to another. It is possible to start 
at any One point in memory and follow a perhaps labyrinthine path to 
reach any other point. Information is retrieved by tracing through the 
network of interconnections to the place where it is stored. 

Retrievability is influenced by the number of locations in which 
information is stored and the number and strength of pathways from 
this information to other concepts that might be activated by incoming 
information. The more frequently a path is followed, the stronger that 
path becomes and the more readily available the information located 
along that path. If one has not thought of a subject for some time, it 
may be difficult to recall details. After thinking our way back into the 
appropriate context and finding the general location in our memory, 
the interconnections become more readily available. We begin to 
remember names, places, and events that had seemed to be forgotten. 

Once people have started thinking about a problem one way, the 
same mental circuits or pathways get activated and strengthened each 
time they think about it. This facilitates the retrieval of information. 
These same pathways, however, also become the mental ruts that 
make it difficult to reorganize the information mentally so as to see it 
from a different perspective. That explains why, in the previous chap- 
ter, once you saw the picture of the old woman it was difficult to see 
the young woman, or vice versa. A subsequent chapter will consider 
ways of breaking out of mental ruts. 

One useful concept of memory organization is what some cogni- 
tive psychologists call a “schema.” A schema is any pattern of rela- 
tionships among data stored in memory. It is any set of nodes and 
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links between them in the spider web of memory that hang together so 
strongly that they can be retrieved and used more or less as a single 
unit. 

For example, a person may have a schema for a bar that when 
activated immediately makes available in memory knowledge of the 
properties of a bar and what distinguishes a bar, say, from a tavern. It 
brings back memories of specific bars that may in turn stimulate 
memories of thirst, guilt, or other feelings or circumstances. People 
also have schemata (plural for schema) for abstract concepts such as a 
socialist economic system and what distinguishes it from a capitalist 
Or communist system. Schemata for phenomena such as success or 
failure in making an accurate intelligence estimate will include links 
to those elements of memory that explain typical causes and implica- 
tions of success or failure. There must also be schemata for processes 
that link memories of the various steps involved in long division, 
regression analysis, or simply making inferences from evidence and 
writing an intelligence report. 

Any given point in memory may be connected to many different 
overlapping schemata. This system is highly complex and not well 
understood. 

This conception of a schema is so general that it begs many 
important questions of interest to memory researchers, but it is the 
best that can be done given the current state of knowledge. It serves 
the purpose of emphasizing that memory does have structure. It also 
shows that how knowledge is connected in memory is critically 
important in determining what information is retrieved in response to 
any stimulus and how that information is used in reasoning. 

Concepts and schemata stored in memory exercise a powerful 
influence on the formation of perceptions from sensory data. Recall 
the experiment discussed in the previous chapter in which test subjects 
were exposed very briefly to playing cards that had been doctored so 
that some hearts were black and spades red. When retained in SIS for 
a fraction of a second, the spades were indeed red. In the course of 
interpreting the sensory impression and transferring it to STM, how- 
ever, the spades became black because the memory system has no 
readily available schema for a red spade to be matched against the 
sensory impression. If information does not fit into what people know, 
or think they know, they have great difficulty processing it. 
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The content of schemata in memory is a principal factor distin- 
guishing stronger from weaker analytical ability. This is aptly illus- 
trated by an experiment with chess players. When chess grandmasters 
and masters and ordinary chess players were given five to 10 seconds 
to note the position of 20 to 25 chess pieces placed randomly on a 
chess board, the masters and ordinary players were alike in being able 
to remember the places of only about six pieces. If the positions of the 
pieces were taken from an actual game (unknown to the test sub- 
jects), however, the grandmasters and masters were usually able to 
reproduce almost all the positions without error, while the ordinary 
players were still able to place correctly only a half-dozen pieces. "' 

That the unique ability of the chess masters did not result from a 
pure feat of memory is indicated by the masters’ inability to perform 
better than ordinary players in remembering randomly placed posi- 
tions. Their exceptional performance in remembering positions from 
actual games stems from their ability to immediately perceive pat- 
terns that enable them to process many bits of information together as 
a single chunk or schema. The chess master has available in long-term 
memory many schemata that connect individual positions together in 
coherent patterns. When the position of chess pieces on the board cor- 
responds to a recognized schema, it is very easy for the master to 
remember not only the positions of the pieces, but the outcomes of 
previous games in which the pieces were in these positions. Similarly, 
the unique abilities of the master analyst are attributable to the sche- 
mata in long-term memory that enable the analyst to perceive patterns 
in data that pass undetected by the average observer. 

Getting Information Into and Out of Long-Term Memory. \t 
used to be that how well a person learned something was thought to 
depend upon how long it was kept in short-term memory or the num- 
ber of times they repeated it to themselves. Research evidence now 
suggests that neither of these factors plays the critical role. Continu- 
Ous repetition does not necessarily guarantee that something will be 
remembered. The key factor in transferring information from short- 
term to long-term memory is the development of associations between 
the new information and schemata already available in memory. This, 
in turn, depends upon two variables: the extent to which the informa- 
tion to be learned relates to an already existing schema, and the level 
of processing given to the new information. 


A. D. deGroot, Thought and Choice in Chess (The Hague: Mouton, 1965) cited by Herbert A. 
Simon, “How Big Is a Chunk?” Science, Vol. 183 (1974), p. 487. 


Take one minute to try to memorize the following items from a 
shopping list: bread, eggs, butter, salami, corn, lettuce, soap, jelly, 
chicken, and coffee. Chances are, you will try to burn the words into 
your mind by repeating them over and over. Such repetition, or main- 
tenance rehearsal, is effective for maintaining the information in STM, 
but is an inefficient and often ineffective means of transferring it to 
LTM. The list is difficult to memorize because it does not correspond 
with any schema already in memory. 

The words are familiar, but you do not have available in mem- 
ory a schema that connects the words in this particular group to each 
other. If the list were changed to juice, cereal, milk, sugar, bacon, 
eggs, toast, butter, jelly, and coffee, the task would be much easier 
because the data would then correspond with an existing schema— 
items commonly eaten for breakfast. Such a list can be assimilated to 
your existing store of knowledge with little difficulty, just as the chess 
master rapidly assimilates the positions of many chessmen. 

Depth of processing is the second important variable in deter- 
mining how well information is retained. Depth of processing refers to 
the amount of effort and cognitive capacity employed to process infor- 
mation, and the number and strength of associations that are thereby 
forged between the data to be learned and knowledge already in mem- 
ory. In experiments to test how well people remember a list of words, 
test subjects might be asked to perform different tasks that reflect dif- 
ferent levels of processing. The following illustrative tasks are listed 
in order of the depth of mental processing required: say how many let- 
ters there are in each word on the list, give a word that rhymes with 
each word, make a mental image of each word, make up a story that 
incorporates each word. 

It turns out that the greater the depth of processing, the greater 
the ability to recall words on a list. This result holds true regardless of 
whether the test subjects are informed in advance that the purpose of 
the experiment is to test them on their memory. Advising test subjects 
to expect a test makes almost no difference in their performance, pre- 
sumably because it only leads them to rehearse mation in 
short-term memory, which is ineffective as compared with other forms 
of processing. 
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There are three ways in which information may be learned or 
committed to memory: by rote, assimilation, or use of a mnemonic 
device. Each of these procedures is discussed below. * 

By Rote. Material to be learned is repeated verbally with suffi- 
cient frequency that it can later be repeated from memory without use 
of any memory aids. When information is learned by rote, it forms a 
separate schema not closely interwoven with previously held knowl- 
edge. That is, the mental processing adds little by way of elaboration 
to the new information, and the new information adds little to the 
elaboration of existing schemata. Learning by rote is a brute force 
technique. It seems to be the least efficient way of remembering. 

By Assimilation. Information is learned by assimilation when the 
structure or substance of the information fits into some memory 
schema already possessed by the learner. The new information is 
assimilated to or linked to the existing schema and can be retrieved 
readily by first accessing the existing schema and then reconstructing 
the new information. Assimilation involves learning by comprehen- 
sion and is, therefore, a desirable method, but it can only be used to 
learn information that is somehow related to our previous experience. 

By Using A Mnemonic Device. A mnemonic device is any means 
of organizing or encoding information for the purpose of making it 
easier to remember. A high school student cramming for a geography 
test might use the acronym “HOMES” as a device for remembering 
the first letter of each of the Great Lakes—Huron, Ontario, etc. 

To learn the first grocery list of disconnected words, you would 
create some structure for linking the words to each other and/or to 
information already in LTM. You might imagine yourself shopping or 
putting the items away and mentally picture where they are located on 
the shelves at the market or in the kitchen. Or you might imagine a 
story concerning one or more meals that include all these items. Any 
form of processing information in this manner is a more effective aid 
to retention than rote repetition. Even more effective systems for 
quickly memorizing lists of names or words have been devised by var- 
ious memory experts, but these require some study and practice in 
their use. 

Mnemonic devices are useful for remembering information that 
does not fit any appropriate conceptual structure or schema already in 
memory. They work by providing a simple, artificial structure to 


° This discussion draws on Francis §. Bellezza, “Mnemonic Devices: Classification, Characteris- 
tics, and Criteria” (Athens, Ohio: Ohio University, pre-publication manuscript, January 1980). 


which the information to be learned is then linked. The mnemonic 
device supplies the mental “file categories” that ensure retrievability 
of information. To remember, first recall the mnemonic device, then 
access the desired information. 


Memory and Intelligence Analysis 


An analyst's memory provides continuous input into the analyti- 
cal process. This input is of two types—additional factual informa- 
tion on historical background and context, and schemata the analyst 
uses to determine the meaning of newly acquired information. Infor- 
mation from memory may force itself on the analyst’s awareness 
without any deliberate effort by the analyst to remember; or, recall of 
the information may require considerable time and strain. In either 
case, anything that influences what information is remembered or 
retrieved from memory also influences intelligence analysis. 

Judgment is the joint product of the available information and 
what the analyst brings to the analysis of this information. An experi- 
ment documenting differences between chess masters and ordinary 
chess players was noted earlier. Similar research with medical doctors 
diagnosing illness indicates that differences between stronger and 
weaker performers are to be found in the organization of information 
and experience in long-term memory.** The same presumably holds 
true for intelligence analysts. Substantive knowledge and analytical 
experience determine the store of memories and schemata the analyst 
draws upon to generate and evaluate hypotheses. The key is not a sim- 
ple ability to recall facts, but the ability to recall patterns that relate 
facts to each other and to broader concepts—and to employ proce- 
dures that facilitate this process. 


Stretching the Limits of Working Memory 

Limited information is available on what is commonly thought 
of as “working memory”—the collection of information that an ana- 
lyst holds in the forefront of the mind as he or she does analysis. The 
general concept of working memory seems clear from personal intro- 
spection. In writing this chapter, | am very conscious of the con- 
straints on my ability to keep many pieces of information in mind 


Arthur S. Elstein, Lee S. Shulman & Sarah A. Sprafka, Medical Problem Solving: An Analysis 
of Clinical Reasoning (Cambridge, MA: Harvard University Press, 1978), p. 276 


while experimenting with ways to organize this information and seek- 
ing words to express my thoughts. To help offset these limits on my 
working memory, I have accumulated a large number of written notes 
containing ideas and half-written paragraphs. Only by using such 
external memory aids am | able to cope with the volume and com- 
plexity of the information I want to use. 

A well-known article written over 40 years ago, titled “The 
Magic Number Seven—Plus or Minus Two,” contends that seven— 
plus or minus two—is the number of things people can keep in their 
head all at once. * That limitation on working memory is the source of 
many problems. People have difficulty grasping a problem in all its 
complexity. This is why we sometimes have trouble making up our 
minds. For example, we think first about the arguments in favor, and 
then about the arguments against, and we can’t keep all those pros and 
cons in our head at the same time to get an overview of how they bal- 
ance off against each other. 

The recommended technique for coping with this limitation of 
working memory is called externalizing the problem—getting it out of 
one’s head and down on paper in some simplified form that shows the 
main elements of the problem and how they relate to each other. 
Chapter 7, “Structuring Analytical Problems,’ discusses ways of 
doing this. They all involve breaking down a problem into its compo- 
nent parts and then preparing a simple “model” that shows how the 
parts relate to the whole. When working on a small part of the prob- 
lem, the model keeps one from losing sight of the whole. 

A simple model of an analytical problem facilitates the assimila- 
tion of new information into long-term memory; it provides a struc- 
ture to which bits and pieces of information can be related. The model 
defines the categories for filing information in memory and retrieving 
it on demand. In other words, it serves as a mnemonic device that pro- 
vides the hooks on which to hang information so that it can be found 
when needed. 

The model is initially an artificial construct, like the previously 
noted acronym “HOMES.” With usage, however, it rapidly becomes 
an integral part of one’s conceptual structure—the set of schemata 
used in processing information. At this point, remembering new infor- 
mation occurs by assimilation rather than by mnemonics. This 


“George A. Miller, “The Magical Number Seven—Pilus or Minus Two: Some Limits on our 
Capacity for Processing Information.” The Psychological Review, Vol. 63, No. 2 (March 1956). 


enhances the ability to recall and make inferences from a larger vol- 
ume of information in a greater variety of ways than would otherwise 
be possible. 

“Hardening of the Categories”. Memory processes tend to work 
with generalized categories. If people do not have an appropriate cate- 
gory for something, they are unlikely to perceive it, store it in mem- 
ory, or be able to retrieve it from memory later. If categories are 
drawn incorrectly, people are likely to perceive and remember things 
inaccurately. When information about phenomena that are different in 
important respects nonetheless gets stored in memory under a single 
concept, errors of analysis may result. For example, many observers 
of international affairs had the impression that Communism was a 
monolithic movement, that it was the same everywhere and con- 
trolled from Moscow. All Communist countries were grouped 
together in a single, undifferentiated category called “international 
Communism” or “the Communist bloc.” In 1948, this led many in the 
United States to downplay the importance of the Stalin-Tito split. 
According to one authority, it “may help explain why many Western 
minds, including scholars, remained relatively blind to the existence 
and significance of Sino-Soviet differences long after they had been 
made manifest in the realm of ideological formulae.” 

“Hardening of the categories” is a common analytical weakness. 
Fine distinctions among categories and tolerance for ambiguity con- 
tribute to more effective analysis. 

Things That Influence What Is Remembered. Factors that influ- 
ence how information is stored in memory and that affect future 
retrievability include: being the first-stored information on a given 
topic, the amount of attention focused on the information, the credibil- 
ity of the information, and the importance attributed to the informa- 
tion at the moment of storage. By influencing the content of memory, 
all of these factors also influence the outcome of intelligence analysis. 

Chapter 12 on “Biases in Estimating Probabilities” describes 
how availability in memory influences judgments of probability. The 
more instances a person can recall of a phenomenon, the more proba- 
ble that phenomenon seems to be. This is true even though ability to 
recall past examples is influenced by vividness of the information, 
how recently something occurred, its impact upon one’s personal wel- 
fare, and many other factors unrelated to the actual probability of the 
phenomenon. 


“ Robert Tucker, “Communist Revolutions, National Cultures, and the Divided Nations.” Studies 
in Comparative Communism (Autumn 1974), 235-245. 
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Memory Rarely Changes Retroactively. Analysis often receive 
new information that should, logically, cause them to reevaluate the 
credibility or significance of previous information. Ideally, the earlier 
information should then become either more salient and readily avail- 
able in memory, or less so. But it does not work that way. Unfortu- 
nately, memories are seldom reassessed or reorganized retroactively in 
response to new information. For example, information that is dis- 
missed as unimportant or irrelevant because it did not fit an analyst's 
expectations does not become more memorable even if the analyst 
changes his or her thinking to the point where the same information, 
received today, would be recognized as very significant. 


Memory Can Handicap as Well as Help 

Understanding how memory works provides some insight into 
the nature of creativity, openness to new information, and breaking 
mind-sets. All involve spinning new links in the spider web of mem- 
ory—links among facts, concepts, and schemata that previously were 
not connected or only weakly connected. 

Training courses for intelligence analysts sometimes focus on 
trying to open up an analyst's established mind-set, to get him or her 
to see problems from different perspectives in order to give a fairer 
shake to alternative explanations. More often than not, the reaction of 
experienced analysts is that they have devoted 20 years to developing 
their present mind-set, that it has served them well, and that they see 
no need to change it. Such analysts view themselves, often accurately, 
as comparable to the chess masters. They believe the information 
embedded in their long-term memory permits them to perceive pat- 
terns and make inferences that are beyond the reach of other observ- 
ers. In one sense, they are quite correct in not wanting to change; it is, 
indeed, their existing schemata or mind-set that enables them to 
achieve whatever success they enjoy in making analytical judgments. 

There is, however, a crucial difference between the chess master 
and the master intelligence analyst. Although the chess master faces a 
different opponent in each match, the environment in which each con- 
test takes place remains stable and unchanging: the permissible moves 
of the diverse pieces are rigidly determined, and the rules cannot be 
changed without the master’s knowledge. Once the chess master 
develops an accurate schema, there is no need to change it. The intel- 
ligence analyst, however, must cope with a rapidly changing world. 
Many countries that previously were US adversaries are now our for- 
mal or de facto allies. The American and Russian governments and 
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societies are not the same today as they were 20 or even 10 or five 
years ago. Schemata that were valid yesterday may no longer be func- 
tional tomorrow. 

Learning new schemata often requires the unlearning of existing 
ones, and this is exceedingly difficult. It is always easier to learn a 
new habit than to unlearn an old one. Schemata in long-term memory 
that are so essential to effective analysis are also the principal source 
of inertia in recognizing and adapting to a changing environment. 
Chapter 6, “Keeping an Open Mind,” identifies tools for dealing with 
this problem. 


PART II—TOOLS FOR THINKING 
Chapter 4 


Strategies for Analytical Judgment: 
Transcending the Limits of 
Incomplete Information 


When intelligence analysts make thoughtful analytical judg- 
ments, how do they do it? In seeking answers to this question, this 
chapter discusses the strengths and limitations of situational logic, 
theory, comparison, and simple immersion in the data as Strategies 
for the generation and evaluation of hypotheses. The final section dis- 
cusses alternative strategies for choosing among hypotheses. One 
strategy too often used by intelligence analysts is described as “satis- 
ficing”—choosing the first hypothesis that appears good enough 
rather than carefully identifying all possible hypotheses and determin- 
ine which is most consistent with the evidence.” 


se te eee eee SESE ESE 


Intelligence analysts should be self-conscious about their reason- 
ing process. They should think about how they make judgments and 
reach conclusions, not just about the judgments and conclusions them- 
selves. Webster's dictionary defines judgment as arriving at a “deci- 
sion or conclusion on the basis of indications and probabilities when 
the facts are not clearly ascertained.” Judgment is what analysts use 
to fill gaps in their knowledge. It entails going beyond the available 
information and is the principal means of coping with uncertainty. It 
always involves an analytical leap, from the known into the uncertain. 


* An earher version of this chapter was published as an unclassified article on Studies in 
Intelligence m 1981, under the title “Strategies for Analytical Judgment.” 
Webster's New Internanonal Dictionary, unabridged, 1954 
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Judgment is an integral part of all intelligence analysis. While 
the optimal goal of intelligence collection is complete knowledge, this 
goal is seldom reached in practice. Almost by definition of the intelli- 
gence mission, intelligence issues involve considerable uncertair ty. 
Thus, the analyst is commonly working with incomplete, ambiguous, 
and often contradictory data. The intelligence analyst’s function might 
be described as transcending the limits of incomplete information 
through the exercise of analytical judgment. 

The ultimate nature of judgment remains a mystery. It is possi- 
ble, however, to identify diverse strategies that analysts employ to pro- 
cess information as they prepare to pass judgment. Analytical 
strategies are important because they influence the data one attends to. 
They determine where the analyst shines his or her searchlight, and 
this inevitably affects the outcome of the analytical process. 


Strategies for Generating and Evaluating Hypotheses 


This book uses the term hypothesis in its broadest sense as a 
potential explanation or conclusion that is to be tested by collecting 
and presenting evidence. Examination of how analysts generate and 
evaluate hypotheses identifies three principal strategies—the applica- 
tion of theory, situational logic, and comparison—each of which is 
discussed at some length below. A “non-strategy,” immersion in the 
data and letting the data speak for themselves, is also discussed. This 
list of analytical strategies is not exhaustive. Other strategies might 
include, for example, projecting one’s own psychological needs onto 
the data at hand, but this discussion is not concerned with the pathol- 
ogy of erroneous judgment. Rather, the goal is to understand the sev- 
eral kinds of careful, conscientious analysis one would hope and 
expect to find among a cadre of intelligence analysts dealing with 
highly complex issues. 


Situational Logic 

This is the most common operating mode for intelligence ana- 
lysts. Generation and analysis of hypotheses start with consideration 
of concrete elements of the current situation, rather than with broad 
generalizations that encompass many similar cases. The situation is 


regarded as one-of-a-kind, so that it must be understood in terms of its 
own unique logic, rather than as one example of a broad class of com- 
parable events. 

Starting with the known facts of the current situation and an 
understanding of the unique forces at work at that particular time and 
place, the analyst seeks to identify the logical antecedents or conse- 
quences of this situation. A scenario is developed that hangs together 
as a plausible narrative. The analyst may work backwards to explain 
the origins or causes of the current situation or forward to estimate the 
future outcome. 

Situational logic commonly focuses on tracing cause-effect rela- 
tionships or, when dealing with purposive behavior, means-ends rela- 
tionships. The analyst identifies the goals being pursued and explains 
why the foreign actor(s) believe certain means will achieve those 
goals. 

Particular strengths of situational logic are its wide applicability 
and ability to integrate a large volume of relevant detail. Any situa- 
tion, however unique, may be analyzed in this manner. 

Situational logic as an analytical strategy also has two principal 
weaknesses. One is that it is so difficult to understand the mental and 
bureaucratic processes of foreign leaders and governments. To see the 
options faced by foreign leaders as these leaders see them, one must 
understand their values and assumptions and even their mispercep- 
tions and misunderstandings. Without such insight, interpreting for- 
eign leaders’ decisions or forecasting future decisions is often little 
more than partially informed speculation. Too frequently, foreign 
behavior appears “irrational” or “not in their own best interest.” Such 
conclusions often indicate analysts have projected American values 
and conceptual frameworks onto the foreign leaders and societies, 
rather than understanding the logic of the situation as it appears to 
them. 

The second weakness is that situational logic fails to exploit the 
theoretical knowledge derived from study of similar phenomena in 
other countries and other time periods. The subject of national sepa- 
ratist movements illustrates the point. Nationalism is a centuries-old 
problem, but most Western industrial democracies have been consid- 
ered well-integrated national communities. Even so, recent years have 
seen an increase in pressures from minority ethnic groups seeking 
independence or autonomy. Why has this phenomenon occurred 


recently in Scotland, southern France and Corsica, Quebec, parts of 
Belgium, and Spain—as well as in less stable Third World countries 
where it might be expecied? 

Dealing with this topic in a logic-of-the-situation mode, a coun- 
try analyst would examine the diverse political, economic, and social 
groups whose interests are at stake in the country. Based on the rela- 
live power positions of these groups, the dynamic interactions among 
them, and anticipated trends or developments that might affect the 
future positions of the interested parties, the analyst would seek to 
identify the driving forces that will determine the eventual outcome. 

It is quite possible to write in this manner a detailed and seem- 
ingly well-informed study of a separatist movement in a single coun- 
try while ignoring the fact that ethnic conflict as a generic 
phenomenon has been the subject of considerable theoretical study. 
By studying similar phenomena in many countries, one can generate 
and evaluate hypotheses concerning root causes that may not even be 
considered by an analyst who is dealing only with the logic of a single 
situation. For example, to what extent does the resurgence of long- 
dormant ethnic sentiments stem from a reaction against the cultural 
homogenization that accompanies modern mass communications sys- 
tems? 

Analyzing many examples of a similar phenomenon, as dis- 
cussed below, enables one to probe more fundamental causes than 
those normally considered in logic-of-the-situation analysis. The prox- 
imate causes identified by situational logic appear, from the broader 
perspective of theoretical analysis, to be but symptoms indicating the 
presence of more fundamental causal factors. A better understanding 
of these fundamental causes is critical to effective forecasting, espe- 
cially over the longer range. While situational logic may be the best 
approach to estimating short-term developments, a more theoretical 
approach is required as the analytical perspective moves further into 
the future. 


Applying Theory 

Theory is an academic term not much in vogue in the Intelli- 
gence Community, but it is unavoidable in any discussion of analytical 
judgment. In one popular meaning of the term, “theoretical” is associ- 
ated with the terms “impractical” and “unrealistic”. Needless to say, it 
is used here in a quite different sense. 

A theory is a generalization based on the study of many exam- 
ples of some phenomenon. It specifies that when a given set of condi- 
tions arises, certain other conditions will follow either with certainty 
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or with some degree of probability. In other words, conclusions are 
judged to follow from a set of conditions and a finding that these con- 
ditions apply in the specific case being analyzed. For example, Tur- 
key is a developing country in a precarious strategic position. This 
defines a set of conditions that imply conclusions concerning the role 
of the military and the nature of political processes in that country, 
because analysts have an implicit if not explicit understanding of how 
these factors normally relate. 

What academics refer to as theory is really only a more explicit 
version of what intelligence analysts think of as their basic under- 
standing of how individuals, institutions, and political systems nor- 
mally behave. 

There are both advantages and drawbacks to applying theory in 
intelligence analysis. One advantage is that “theory economizes 
thought.” By identifying the key elements of a problem, theory 
enables an analyst to sort through a mass of less significant detail. 
Theory enables the analyst to see beyond today’s transient develop- 
ments, to recognize which trends are superficial and which are signifi- 
cant, and to foresee future developments for which there is today little 
concrete evidence. 

Consider, for example, the theoretical proposition that economic 
development and massive infusion of foreign ideas in a feudal society 
lead to political instability. This proposition seems well established. 
When applied to Saudi Arabia, it suggests that the days of the Saudi 
monarchy are numbered, although analysts of the Saudi scene using 
situational logic find little or no current evidence of a meaningful 
threat to the power and position of the royal family. Thus, the applica- 
tion of a generally accepted theoretical proposition enables the ana- 
lyst to forecast an outcome for which the “hard evidence” has not yet 
begun to develop. This is an important strength of theoretical analysis 
when applied to real-world problems. 

Yet this same example also illustrates a common weakness in 
applying theory to analysis of political phenomena. Theoretical propo- 
sitions frequently fail to specify the time frame within which develop- 
ments might be anticipated to occur. The analytical problem with 
respect to Saudi Arabia is not so much whether the monarchy will 
eventually be replaced, as when or under what conditions this might 
happen. Further elaboration of the theory relating economic develop- 
ment and foreign ideas to political instability in feudal societies would 
identify early warning indicators that analysts might look for. Such 


indicators would guide both intelligence collection and analysis of 
sociopolitical and socioeconomic data and lead to hypotheses con- 
cerning when or under what circumstances such an event might occur. 

But if theory enables the analyst to transcend the limits of avail- 
able data, it may also provide the basis for ignoring evidence thot is 
truly indicative of future events. Consider the following theoretical 
propositions in the light of popular agitation against the Shah of Iran 
in the late 1970s: (1) When the position of an authoritarian ruler is 
threatened, he will defend his position with force if necessary. (2) An 
authoritarian ruler enjoying complete support of effective military and 
security forces cannot be overthrown by popular opinion and agita- 
tion. Few would challenge these propositions, yet when applied to 
Iran in the late 1970s, they led Iran specialists to misjudge the Shah's 
chances for retaining the peacock throne. Many if not most such spe- 
cialists seemed convinced that the Shah remained strong and that he 
would crack down on dissent when it threatened to get out of control. 
Many persisted in this assessment for several months after the accu- 
mulation of what in retrospect appears to have been strong evidence to 
the contrary. 

Persistence of these assumptions is easily understood in psycho- 
logical terms. When evidence is lacking or ambiguous, the analyst 
evaluates hypotheses by applying his or her general background 
knowledge concerning the nature of political systems and behavior. 
The evidence on the strength of the Shah and his intention to crack 
down on dissidents was ambiguous, but the Iranian monarch was an 
authoritarian ruler, and authoritarian regimes were assumed to have 
certain characteristics, as noted in the previously cited propositions. 
Thus beliefs about the Shah were embedded in broad and persuasive 
assumptions about the nature of authoritarian regimes per se. For an 
analyst who believed in the two aforementioned propositions, it would 
have taken far more evidence, including more unambiguous evidence, 
to infer that the Shah would be overthrown than to justify continued 
confidence in his future.* 


“ Even in retrospect these two propositions still seem valid, which is why some aspects of the 
Shah's fall remain incredible. There are, in principle, three possible reasons why these seemingly 
valid theoretical assumptions failed to generate an accurate estimate on Iran: (1) One or more of 
the initial conditions posited by the theory did not in fact apply---for example, the Shah was not 
really an authoritarian ruler. (2) The theory is only partially valid, in that there are certain circum- 
stances under which it does and does not apply. These limiting conditions need to be specified. (3) 
The theory is basically valid, but one cannot expect 100-percent accuracy from social science the- 
ones. Social science, as distinct from natural science, deals with a probabilistic environment. One 
cannot foresee all the circumstances that might cause an exception to the general rules, so the best 
that can be expected is that the given conditions will lead to the specified outcome most of the time. 
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Figure 4 below illustrates graphically the difference between the- 
ory and situational logic. Situational logic looks at the evidence within 
a single country on multiple interrelated issues, as shown by the col- 
umn highlighted in gray. This is a typical area studies approach. Theo- 
retical analysis looks at the evidence related to a single issue in 
multiple countries, as shown by the row highlighted in gray. This is a 
typical social science approach. 


Figure 4 


Country Country Country Country 


Issue ‘Aged Evidence | Evidence 


Issue Evidence Evidence Evidence Evidence 


Evidence BAdC isa Evidence | Evidence 


Issue 


Issue Evidence BAZ scam Evidence | Evidence 


Situational Logic vs. Theory 


The distinction between theory and situational logic is not as 
clear as it may seem from this graphic, however. Logic-of-the-situa- 
tion analysis also draws heavily on theoretical assumptions. How does 
the analyst select the most significant elements to describe the current 
situation, or identify the causes or consequences of these elements, 
without some implicit theory that relates the likelihood of certain out- 
comes to certain antecedent conditions? 

For example, if the analyst estimating the outcome of an 
impending election does not have current polling data, it is necessary 
to look back at past elections, study the campaigns, and then judge 
how voters are likely to react to the current campaigns and to events 
that influence voter attitudes. In doing so, the analyst operates from a 
set of assumptions about human nature and what drives people and 
groups. These assumptions form part of a theory of political behavior, 
but it is a different sort of theory than was discussed under theoretical 
analysis. It does not illuminate the entire situation, but only a small 
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increment of the situation, and it may not apply beyond the specific 
country of concern. Further, it is much more likely to remain implicit, 
rather than be a focal point of the analysis. 


Comparison with Historical Situations 

A third approach for going beyond the available information is 
comparison. An analyst seeks understanding of current events by 
comparing them with historical precedents in the same country, or 
with similar events in other countries. Analogy is one form of com- 
parison. When an historical situation is deemed comparable to cur- 
rent circumstances, analysts use their understanding of the historical 
precedent to fill gaps in their understanding of the current situation. 
Unknown elements of the present are assumed to be the same as 
known elements of the historical precedent. Thus, analysts reason that 
the same forces are at work, that the outcome of the present situation 
is likely to be similar to the outcome of the historical situation, or that 
a certain policy is required in order to avoid the same outcome as in 
the past. 

Comparison differs from situational logic in that the present situ- 
ation is interpreted in the light of a more or less explicit conceptual 
model that is created by looking at similar situations in other times or 
places. It differs from theoretical analysis in that this conceptual 
model is based on a single case or only a few cases, rather than on 
many similar cases. Comparison may also be used to generate theory, 
but this is a more narrow kind of theorizing that cannot be validated 
nearly as well as generalizations inferred from many comparable 
cases. 

Reasoning by comparison is a convenient shortcut, one chosen 
when neither data nor theory are available for the other analytical 
strategies, or simply because it is easier and less time-consuming than 
a more detailed analysis. A careful comparative analysis starts by 
specifying key elements of the present situation. The analyst then 
seeks out one or more historical precedents that may shed light on the 
present. Frequently, however, a historical precedent may be so vivid 
and powerful that it imposes itself upon a person’s thinking from the 
outset, conditioning them to perceive the present primarily in terms of 
its similarity to the past. This is reasoning by analogy. As Robert 
Jervis noted, “historical analogies often precede, rather than follow, a 
careful analysis of a situation.” 


“ Robert Jervis, “Hypotheses on Misperception,” World Politics 20 (April 1968), p. 471. 
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The tendency to relate contemporary events to earlier events as a 
guide to understanding is a powerful one. Comparison helps achieve 
understanding by reducing the unfamiliar to the familiar. In the 
absence of data required for a full understanding of the current situa- 
tion, reasoning by comparison may be the only alternative. Anyone 
taking this approach, however, should be aware of the significant 
potential for error. This course is an implicit admission of the lack of 
sufficient information to understand the present situation in its own 
right, and lack of relevant theory to relate the present situation to 
many other comparable situations 

The difficulty, of course, is in being certain that two situations 
are truly comparable. Because they are equivalent in some respects, 
there is a tendency to reason as though they were equivalent in all 
respects, and to assume that the current situation will have the same or 
similar outcome as the historical situation. This is a valid assumption 
only when based on in-depth analysis of both the current situation and 
the historical precedent to ensure that they are actually comparable in 
all relevant respects. 

In a short book that ought to be familiar to all intelligence ana- 
lysts, Ernest May traced the impact of historical analogy on US for- 
eign policy.“’ He found that because of reasoning by analogy, US 
policymakers tend to be one generation behind, determined to avoid 
the mistakes of the previous generation. They pursue the policies that 
would have been most appropriate in the historical situation but are 
not necessarily well adapted to the current one. 

Policymakers in the 1930s, for instance, viewed the interna- 
tional situation as analogous to that before World War |. Conse- 
quently, they followed a policy of isolation that would have been 
appropriate for preventing American involvement in the first World 
War but failed to prevent the second. Communist aggression after 
World War Il was seen as analogous to Nazi aggression, leading to a 
policy of containment that could have prevented World War II. 

More recently, the Vietnam analogy has been used repeatedly 
over many years to argue against an activist US foreign policy. For 
example, some used the Vietnam analogy to argue against US partici- 
pation in the Gulf War—a flawed analogy because the operating ter- 
rain over which battles were fought was completely different in 
Kuwait/Iraq and much more in our favor there as compared with Viet- 
nam. 


“ Ernest May, ‘Lessons’ of the Past: The Use and Misuse of History in American Foreign Policy 
(New York: Oxford University Press, 1973). 
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May argues that policymakers often perceive problems in terms 
of analogies with the past, but that they ordinarily use history badly: 


When resorting to an analogy, they tend to seize upon the first 
that comes to mind. They do not research more widely. Nor do 
they pause to analyze the case, test its fitness, or even ask in what 
ways it might be misieading.*' 


As compared with policymakers, intelligence analysts have more 
time available to “analyze rather than analogize.” Intelligence ana- 
lysts tend to be good historians, with a large number of historical pre- 
cedents available for recall. The greater the number of potential 
analogues an analyst has at his or her disposal, the greater the likeli- 
hood of selecting an appropriate one. The greater the depth of an ana- 
lyst’s knowledge, the greater the chances the analyst will perceive the 
differences as well as the similarities between two situations. Even 
under the best of circumstances, however, inferences based on com- 
parison with a single analogous situation probably are more prone to 
error than most other forms of inference. 

The most productive uses of comparative analysis are to suggest 
hypotheses and to highlight differences, not to draw conclusions. 
Comparison can suggest the presence or the influence of variables that 
are not readily apparent in the current situation, or stimulate the imag- 
ination to conceive explanations or possible outcomes that might not 
otherwise occur to the analyst. In short, comparison can generate 
hypotheses that then guide the search for additional information to 
confirm or refute these hypotheses. It should not, however, form the 
basis for conclusions unless thorough analysis of both situations has 
confirmed they are indeed comparable. 


Data Immersion 

Analysts sometimes describe their work procedure as immers- 
ing themselves in the data without fitting the data into any precon- 
ceived pattern. At some point an apparent pattern (or answer or 
explanation) emerges spontaneously, and the analyst then goes back to 
the data to check how well the data support this judgmert. According 
to this view, objectivity requires the analyst to suppress any personal 
opinions or preconceptions, so as to be guided only by the “facts” of 
the case. 


* Thid., p. xi 
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To think of analysis in this way overlooks the fact that informa- 
tion cannot speak for itself. The significance of information is always 
a joint function of the nature of the information and the context in 
which it is interpreted. The context is provided by the analyst in the 
form of a set of assumptions and expectations concerning human and 
organizational behavior. These preconceptions are critical determi- 
nants of which information is considered relevant and how it is inter- 
preted. 

Of course there are many circumstances in which the analyst has 
no option but to immerse himself or herself in the data. Obviously, an 
analyst must have a base of knowledge to work with before starting 
analysis. When dealing with a new and unfamiliar subject, the uncriti- 
cal and relatively non-selective accumulation and review of informa- 
tion is an appropriate first step. But this is a process of absorbing 
information, not analyzing it. 

Analysis begins when the analyst consciously inserts himself or 
herself into the process to select, sort, and organize information. This 
selection and organization can only be accomplished according to 
conscious or subconscious assumptions and preconceptions. 

The question is not whether one’s prior assumptions and expec- 
tations influence analysis, but only whether this influence is made 
explicit or remains implicit. The distinction appears to be important. 
In research to determine how physicians make medical diagnoses, the 
doctors who comprised the test subjects were asked to describe their 
analytical strategies. Those who stressed thorough collection of data 
as their principal analytical method were significantly less accurate in 
their diagnoses than those who described themselves as following 
other analytical strategies such as identifying and testing hypothe- 
ses.“ Moreover, the collection of additional data through greater thor- 
oughness in the medical history and physical examination did not lead 
to increased diagnostic accuracy.*’ 

One might speculate that the analyst who seeks greater objectiv- 
ity by suppressing recognition of his or her own subjective input actu- 
ally has less valid input to make. Objectivity is gained by making 
assumptions explicit so that they may be examined and challenged, 
not by vain efforts to eliminate them from analysis. 


© Arthur S. Elstein, Lee S. Shulman, and Sarah A. Sprafka, Medical Problem Solving: An Analysis 
of Clinical Reasoning (Cambridge, MA: Harvard University Press, 1978), p. 270 

* Ibid... p. 281. For more extensive discussion of the value of additional information, see Chapter 
5. “Do You Really Need More Information?” 


Relationships Among Strategies 


No one strategy is necessarily better than the others. In order to 
generate all relevant hypotheses and make maximum use of all poten- 
tially relevant information, it would be desirable to employ all three 
Strategies at the early hypothesis generation phase of a research 
project. Unfortunately, analysts commonly lack the inclination or time 
to do so. 

Different analysts have different analytical habits and prefer- 
ences for analytical strategy. As a broad generalization that admits 
numerous exceptions, analysts trained in area studies or history tend 
to prefer situational logic, while those with a strong social science 
background are more likely to bring theoretical and comparative 
insights to bear on their work. The Intelligence Community as a 
whole is far stronger in situational logic than in theory. In my judg- 
ment, intelligence analysts do not generalize enough, as opposed to 
many academic scholars who generalize too much. This is especially 
true in political analysis, and it is not entirely due to unavailability of 
applicable political theory. Theoretical insights that are available are 
often unknown to or at least not used by political intelligence analysts. 

Differences in analytical strategy may cause fundamental differ- 
ences in perspective between intelligence analysts and some of the 
policymakers for whom they write. Higher ievel officials who are not 
experts on the subject at issue use far more theory and comparison 
and less situational logic than intelligence analysts. Any policymaker 
or other senior manager who lacks the knowledge base of the special- 
ist and does not have time for detail must, of necessity, deal with 
broad generalizations. Many decisions must be made, with much less 
time to consider each of them than is available to the intelligence ana- 
lyst. This requires the policymaker to take a more conceptual 
approach, to think in terms of theories, models, or analogies that sum- 
marize large amounts of detail. Whether thi: represents sophistication 
or oversimplification depends upon the individual case and, perhaps, 
whether one agrees or disagrees with the judgments made. In any 
event, intelligence analysts would do well to take ‘his phenomenon 
into account when writing for their consumers. 


Strategies for Choice Among Hypotheses 


A systematic analytical process requires selection among alter- 
native hypotheses, and it is here that analytical practice often divei ges 
significantly from the ideal and from the canons of scientific method. 
The ideal is to generate a full set of hypotheses, systematically evalu- 
ate each hypothesis, and then identify the hypothesis that provides the 
best fit to the data. Scientific method, for its part, requires that one 
seek to disprove hypotheses rather than confirm them. 

In practice, other strategies are commonly employed. Alexander 
George has identified a number of less-than-optimal strategies for 
making decisions in the face of incomplete information and multiple, 
competing values and goals. While George conceived of these strate- 
gies as applicable to how decisionmakers choose among alternative 
policies, most also apply to how intelligence analysts might decide 
among alternative analytical hypotheses. 

The relevant strategies George identified are: 


* “Satisficing”—selecting the first identified alternative that 
appears “good enough” rather than examining all alternatives to 
determine which is “best.” 


* Incrementalism—focusing on a narrow range of alternatives 
representing marginal change, without considering the need for 
dramatic change from an existing position. 


¢ Consensus—opting for the alternative that will elicit the great- 
est agreement and support. Simply telling the boss what he or 
she wants to hear is one version of this. 


* Reasoning by analogy—choosing the alternative that appears 
most likely to avoid some previous error or to duplicate a pre- 
vious success. 


* Relying on a set of principles or maxims that distinguish a 
“good” from a “bad” alternative.“ 


The intelligence analyst has another tempting option not avail- 
able to the policymaker: to avoid judgment by simply describing the 
current situation, identifying alternatives, and letting the intelligence 


“ Alexander George, Presidential Decisionmaking in Foreign Policy: The Effective Use of Infor- 
mation and Advice (Boulder, CO: Westview Press, 1980), Chapter 2 
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consumer make the judgment about which alternative is most likely. 
Most of these strategies are not discussed here. The following para- 
graphs focus only on the one that seems most prevalent in intelli- 
gence analysis. 


“Satisficing” 

I would suggest, based on personal experience and discussions 
with analysts, that most analysis is conducted in a manner very similar 
to the satisficing mode (selecting the first identified alternative that 
appears “good enough”).* The analyst identifies what appears to be 
the most likely hypothesis—that is, the tentative estimate, explanation, 
or description of the situation that appears most accurate. Data are 
collected and organized according to whether they support this tenta- 
tive judgment, and the hypothesis is accepted if it seems to provide a 
reasonable fit to the data. The careful analyst will then make a quick 
review of other possible hypotheses and of evidence not accounted for 
by the preferred judgment to ensure that he or she has not overlooked 
some important consideration. 

This approach has three weaknesses: the selective perception 
that results from focus on a single hypothesis, failure to generate a 
complete set of competing hypotheses, and a focus on evidence that 
confirms rather than disconfirms hypotheses. Each of these is dis- 
cussed below. 

Selective Perception. Tentative hypotheses serve a useful func- 
tion in helping analysts select, organize, and manage information. 
They narrow the scope of the problem so that the analyst can focus 
efficiently on data that are most relevant and important. The hypothe- 
ses serve as organizing frameworks in working memory and thus 
facilitate retrieval of information from memory. In short, they are 
essential elements of the analytical process. But their functional utility 
also entails some cost, because a hypothesis functions as a perceptual 
filter. Analysts, like people in general, tend to see what they are look- 
ing for and to overlook that which is not specifically included in their 
search strategy. They tend to limit the processed information to that 
which is relevant to the current hypothesis. If the hypothesis is incor- 
rect, information may be lost that would suggest a new or modified 
hypothesis. 


“ The concept of “satisficing.” of seeking a satisfactory rather than an optimal solution, was devel- 
oped by Herbert A. Simon and is widely used in the literature on decision analysis 


This difficulty can be overcome by the simultaneous consider- 
ation of multiple hypotheses. This approach is discussed in detail in 
Chapter 8. It has the advantage of focusing attention on those few 
items of evidence that have ‘he greatest diagnostic value in distin- 
guishing among the validity 01 competing hypotheses. Most evidence 
is consistent with several different hypotheses, and this fact is easily 
overlooked when analysts focus on only one hypothesis at a time— 
especially if their focus is on seeking to confirm rather than disprove 
what appears to be the most likely answer. 

Failure To Generate Appropriate Hypotheses. \f tentative hypo- 
theses determine the criteria for searching for information anc judg- 
ing its relevance, it follows that one may overlook the proper answer 
if it is not encompassed within the several hypotheses being consid- 
ered. Researc:: »n hypothesis generation suggests that performance on 
this task is woefully inadequate.” When faced with an analytical prob- 
lem, people are either unable or simply do not take the time to iden- 
tify the full range of potential answers. Analytical performance might 
be significantly enhanced by more deliberate attention to this stage of 
the analytical process. Analysts need to take more time to develop a 
full set of competing hypotheses, using all three of the previously dis- 
cussed strategies—theory, situational logic, and comparison. 

Failure To Consider Diagnosticity of Evidence. \n the absence 
of a complete set of alternative hypotheses, it is not possible to evalu- 
ate the “diagnosticity” of evidence. Unfortunately, many analysts are 
unfamiliar with the concept of diagnosticity of evidence. It refers to 
the extent to which any item of evidence helps the analyst determine 
the relative likelihood of alternative hypotheses. 

To illustrate, a high temperature may have great value in telling a 
doctor that a patient is sick, but relatively little value in determining 
which illness the patient is suffering from. Because a high temperature 
is consistent with so many possible hypotheses about a patient's ill- 
ness, it has limited diagnostic value in determining which illness 
(hypothesis) is the more likely one. 

Evidence is diagnostic when it influences an analyst's judgment 
on the relative likelihood of the various hypotheses. If an item of evi- 
dence seems consistent with all the hypotheses, it may have no diag- 
nostic value at all. It is a common experience to discover that most 
available evidence really is not very helpful, as it can be reconciled 
with all the hypotheses. 


* Charles Gettys et al., Hypothesis Generates: A Final Report on Three Years of Research. Tech- 
mcal Report 15-10-80. University of Oklahoma, Decision Processes Laboratory, 1980 
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Failure To Reject Hypotheses 

Scientific method is based on the principle of rejecting hypothe- 
ses, while tentatively accepting only those hypotheses that cannot be 
refuted. Intuitive analysis, by comparison, generally concentrates on 
confirming a hypothesis and commonly accords more we}, /ti to evi- 
dence supporting a hypothesis than to evidence that weakens it. Ide- 
ally, the reverse would be true. While analysts usually cannot apply 
the statistical procedures of scientific methodology to test their 
hypotheses, they can and should adopi the conceptual strategy of 
seeking to refute rather than confirm hypotheses. 

There are two aspects to this problem: people do not naturally 
seek disconfirming evidence, and when such evidence is received it 
tends to be discounted. If there is any question about the former, con- 
sider how often people test their political and religious beliefs by 
reading newspapers and books representing an opposing viewpoint. 
Concerning the latter, we have noted in Chapter 2, “Perception: Why 
Can't We See What Is There to Be Seen?” the tendency to accommo- 
date new information to existing images. This is easy to do if informa- 
tion supporting a hypothesis is accepted as valid, while information 
that weakens it is judged to be of questionable reliability or an unim- 
portant anomaly. When information is processed in this manner, it is 
easy to “confirm” almost any hypothesis that one already believes to 
be true. 

Apart from the psychologica! pitfalls involved in seeking confir- 
matory evidence, an important logical point also needs to be consid- 
ered. The logical reasoning underlying the scientific method of 
rejecting hypotheses is that “...no confirming instance of a law is a 
verifying instance, but that any disconfirming instance is a falsifying 
instance.”*’ In other words, a hypothesis can never be proved by the 
enumeration of even a large body of evidence consistent with that 
hypothesis, because the same body of evidence may also be consis- 
tent with other hypotheses. A hypothesis may be disproved, however, 
by citing a single item of evidence that is incompatible with it. 

P. C. Wason conducted a series of experiments to test the view 
that people generally seek confirming rather than disconfirming evi- 
dence.“ The experimental design was based on the above point that 
the validity of a hypothesis can only be tested by seeking to disprove 


“ PC. Wason, “On the Failure to Eliminate Hypotheses in a Conceptual Task,” The Quarterly 
Journal of Experimental Psychology, Vo\. X11, Part 3 (1960). 
“ Wason, ibid. 


it rather than confirm it. Test subjects were given the three-number 
sequence, 2 - 4 - 6, and asked to discover the rule employed to gener- 
ate this sequence. In order to do so, they were permitted to generate 
three-number sequences of their own and to ask the experimenter 
whether these conform to the rule. They were encouraged to generate 
and ask about as many sequences as they wished and were instructed 
to stop only when they believed they had discovered the rule. 

There are, of course, many possible rules that might account for 
the sequence 2 - 4 - 6. The test subjects formulated tentative hypothe- 
ses such as any ascending sequence of even numbers, or any sequence 
separated by two digits. As expected, the test subjects generally took 
the incorrect approach of trying to confirm rather than eliminate such 
hypotheses. To test the hypothesis that the rule was any ascending 
sequence of even numbers, for example, they might ask if the 
sequence 8 - 10 - 14 conforms to the rule. 

Readers who have followed the reasoning to this point will rec- 
ognize that this hypothesis can never be proved by enumerating exam- 
ples of ascending sequences of even numbers that are found to 
conform to the sought-for rule. One can only disprove the hypothesis 
by citing an ascending sequence of odd numbers and learning that 
this, too, conforms to the rule. 

The correct rule was any three ascending numbers, either odd or 
even. Because of their strategy of seeking confirming evidence, only 
six ui the 29 test subjects in Wason’s experiment were correct the first 
time they thought they had discovered the rule. When this same exper- 
iment was repeated by a different researcher for a somewhat different 
purpose, none of the 51 test subjects had the right answer the first 
time they thought they had discovered the rule.” 

In the Wason experiment, the strategy of seeking confirming 
rather than disconfirming evidence was particularly misleading 
because the 2 - 4 - 6 sequence is consistent with such a large number 
of hypotheses. It was easy for test subjects to obtain confirmatory evi- 
dence for almost any hypothesis they tried to confirm. It is important 
to recognize that comparable situations, when evidence is consistent 
with several different hypotheses, are extremely common in intelli- 
gence analysis. 


“ Harold M. Weiss and Patrick A. Knight, “The Utility of Humility: Self-Esteem, Information 
Search, and Problem-Solving Efficiency,” Organizational Behavior and Human Performance, Vol 
25, No. 2 (April 1980), 216-223. 


Consider lists of early warning indicators, for example. They are 
designed to be indicative of an impending attack. Very many of them, 
however, are also consistent with the hypothesis that military move- 
ments are a bluff to exert diplomatic pressure and that no military 
action will be forthcoming. When analysts seize upon only one of 
these hypotheses and seek evidence to confirm it, they will often be 
led astray. 

The evidence available to the intelligence analyst is in one 
important sense different from the evidence available to test subjects 
asked to infer the number sequence rule. The intelligence analyst 
commonly deals with problems in which the evidence has only a 
probabilistic relationship to the hypotheses being considered. Thus it 
is seldom possible to eliminate any hypothesis entirely, because the 
most one can say is that a given hypothesis is unlikely given the 
nature of the evidence, not that it is impossible. 

This weakens the conclusions that can be drawn from a strategy 
aimed at eliminating hypotheses, but it does not in any way justify a 
strategy aimed at confirming them. 

Circumstances and insufficient data often preclude the applica- 
tion of rigorous scientific procedures in intelligence analysis—includ- 


ing, in particular, statistical methods for testing hypotheses. There is, 
however, certainly no reason why the basic conceptual strategy of 
looking for contrary evidence cannot be employed. An optimal analyt- 
ical strategy requires that analysts search for information to discon- 
firm their favorite theories, not employ a satisficing strategy that 
permits acceptance of the first hypothesis that seems consistent with 
the evidence. 


Conclusion 


There are many detailed assessments of intelligence failures, but 
few comparable descriptions of intelligence successes. In reviewing 
the literature on intelligence successes, Frank Stech found many 
examples of success but only three accounts that provide sufficient 
methodological details to shed light on the intellectual processes and 
methods that contributed to the successes. These dealt with successful 


American and British intelligence efforts during World War II to ana- 
lyze German propaganda, predict German submarine movements, and 
estimate future capabilities and intentions of the German Air Force.“ 

Stech notes that in each of these highly successful efforts, the 
analysts employed procedures that “... facilitated the formulation and 
testing against each other of alternative hypothetical estimates of 
enemy intentions. Each of the three accounts stressed this pitting of 
competing hypotheses against the evidence.”*' 

The simultaneous evaluation of multiple, competing hypotheses 
permits a more systematic and objective analysis than is possible 
when an analyst focuses on a single, most-likely explanation or esti- 
mate. The simultaneous evaluation of multiple, competing hypotheses 
entails far greater cognitive strain than examining a single, most-likely 
hypothesis. Retaining multiple hypotheses in working memory and 
noting how each item of evidence fits into each hypothesis add up to a 
formidable cognitive task. That is why this approach is seldom 
employed in intuitive analysis of complex issues. It can be accom- 
plished, however, with the help of simple procedures described in 
Chapter 8, “Analysis of Competing Hypotheses.” 


“ Alexander George, Propaganda Analysis: A Study of Inferences Made From Nazi Propaganda 
in World War II (Evanston, IL: Row, Peterson, 1959), Patrick Beesly, Very Special Intelligence. 
The Story of the Admiralty's Operational Intelligence Center 1939-1945 (London: Hamish Hamil- 
ton, 1977): and R. V. Jones, Wizard War: British Scientific Intelligence 1939-1945 (New York 
Coward, McCann & Geoghegan, 1978). 

Frank J. Stech, Political and Military Intention Estimation: A Taxonometric Analysis, Final 
Report for Office of Naval Research (Bethesda, MD: MATHTECH, Inc., November 1979), p. 283 


Chapter 5 


Do You Really Need 
More Information? 


The difficulties associated with intelligence analysis are often 
attributed to the inadequacy of available information. Thus the US 
Intelligence Community invests heavily in improved intelligence col- 
lection systems while managers of analysis lament the comparatively 
small sums devoted to enhancing analytical resources, improving 
analytical methods, or gaining better understanding of the cognitive 
processes involved in making analytical judgments. This chapter 
questions the often-implicit assumption that lack of information is the 
principal obstacle to accurate intelligence judgmenis.* 
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Using experts in a variety of fields as test subjects, experimental 
psychologists have examined the relationship between the amount of 
information available to the experts, the accuracy of judgments they 
make based on this information, and the experts’ confidence in the 
accuracy of these judgments. The word “information,” as used in this 
context, refers to the totality of material an analyst has available to 
work with in making a judgment. 

Key findings from this research are: 


* Once an experienced analyst has the minimum information 
necessary to make an informed judgment, obtaining additional 
information generally does not improve the accuracy of his or 


© This is an edited version of an article that appeared in Studies in Intelligence, Vol. 23, No. | 
(Spring 1979). That Studies in Intelligence article was later reprinted in H. Bradford Westerfield, 
ed., Inside CIA's Private World: Declassified Articles from the Agency's Internal Journal, 1955. 
1992 (New Haven: Yale University Press, 1995). A slightly different version was published in The 
Bureaucrat, Vol. 8, 1979, under the title “Improving Intelligence Analysis: Some Insights on Data, 
Concepts, and Management in the Intelligence Community.” For this book, portions of the original 
article dealing with improving intelligence analysis have been moved to Chapter 14 on “Improving 
Intelligence Analysis.” 
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her estimates. Additional information does, however, lead the 
analyst to become more confident in the judgment, to the point 
of overconfidence. 


¢ Experienced analysts have an imperfect understanding of what 
information they actually use in making judgments. They are 
unaware of the extent to which their judgments are determined 
by a few dominant factors, rather than by the systematic inte- 
gration of all available information. Analysts actually use much 
less of the available information than they think they do. 


As will be noted below, these experimental findings should not 
necessarily be accepted at face value. For example, circumstances 
exist in which additional information does contribute to more accu- 
rate analysis. However, there also are circumstances in which addi- 
tional information—particularly contradictory information—decreases 
rather than increases an analyst's confidence. 

To interpret the disturbing but not surprising findings from these 
experiments, it is necessary to consider four different types of infor- 
mation and discuss their relative value in contributing to the accuracy 
of analytical judgments. It is also helpful to distinguish analysis in 
which results are driven by the data from analysis that is driven by the 
conceptual framework employed to interpret the data. 

Understanding the complex relationship between amount of 
information and accuracy of judgment has implications for both the 
management and conduct of intelligence analysis. Such an under- 
standing suggests analytical procedures and management initiatives 
that may indeed contribute to more accurate analytical judgments. It 
also suggests that resources needed to attain a better understanding of 
the entire analytical process might profitably be diverted from some of 
the more costly intelligence collection programs. 

These findings have broad relevance beyond the Intelligence 
Community. Analysis of information to gain a better understanding of 
current developments and to estimate future outcomes is an essential 
component of decisionmaking in any field. In fact, the psychological 
experiments that are most relevant have been conducted with experts 
in such diverse fields as medical and psychological diagnosis, stock 
market analysis, weather forecasting, and horserace handicapping. 
The experiments reflect basic human processes that affect analysis of 
any subject. 


One may conduct experiments to demonstrate these phenomena 
in any field in which experts analyze a finite number of identifiable 
and classifiable kinds of information to make judgments or estimates 
that can subsequently be checked for accuracy. The stock market ana- 
lyst, for example, commonly works with information concerning 
price-earnings ratios, profit margins, earnings per share, market vol- 
ume, and resistance and support levels, and it is relatively easy to 
measure quantitatively the accuracy of the resulting predictions. By 
controlling the information made available to a group of experts and 
then checking the accuracy of judgments based on this information, it 
is possible to investigate how people use information to arrive at ana- 
lytical judgments. 


An Experiment: Betting on the Horses 


A description of one such experiment serves to illustrate the pro- 
cedure.*' Eight experienced horserace handicappers were shown a list 
of 88 variables found on a typical past-performance chart—for exam- 
ple, the weight to be carried; the percentage of races in which horse 
finished first, second, or third during the previous year; the jockey’s 
record; and the number of days since the horse’s last race. Each handi- 
capper was asked to identify, first, what he considered to be the five 
most important items of information—those he would wish to use to 
handicap a race if he were limited to only five items of information 
per horse. Each was then asked to select the 10, 20, and 40 most 
important variables he would use if limited to those levels of informa- 
tion. 

At this point, the handicappers were given true data (sterilized so 
that horses and actual races could not be identified) for 40 past races 
and were asked to rank the top five horses in each race in order of 
expected finish. Each handicapper was given the data in increments of 
the 5, 10, 20 and 40 variables he had judged to be most useful. Thus, 
he predicted each race four times—once with each of the four differ- 
ent levels of information. For each prediction, each handicapper 
assigned a value from 0 to 100 percent to indicate degree of confi- 
dence in the accuracy of his prediction. 


* Paul Slovic, “Behavioral Problems of Adhering to a Decision Policy.” unpublished manuscript, 
1973 


When the handicappers’ predictions were compared with the 
actual outcomes of these 40 races, it was clear that average accuracy 
of predictions remained the same regardless of how much informa- 
tion the handicappers had available. Three of the handicappers actu- 
ally showed less accuracy as the amount of information increased, two 
improved their accuracy, and three were unchanged. All, however, 
expressed steadily increasing confidence in their judgments as more 
information was received. This relationship between amount of infor- 
mation, accuracy of the handicappers’ prediction of the first place 
winners, and the handicappers’ confidence in their predictions is 
shown in Figure 5. 

With only five items of information, the handicappers’ confi- 
dence was well calibrated with their accuracy, but they became over- 
confident as additional information was received. 


Figure 5 


CONFIDENCE 


be ACCURACY - ? 


The same relationships among amount of information, accury. ¥. 
and analyst confidence have been confirmed by similar experiments in 
other fields.“ In one experiment with clinical psychologists, a psycho- 
logical case file was divided into four sections representing succes- 
sive chronological periods in the life of a relatively normal individual. 
Thirty-two psychologists with varying levels of experience were asked 
to make judgments on the basis of this information. After reading 
each section of the case file, the psychologists answered 25 questions 
(for which there were known answers) about the personality of the 
subject of the file. As in other experiments, increasing information 
resulted in a strong rise in confidence but a negligible increase in 
accuracy. ~ 

A series of experiments to examine the mental processes of med- 
ical doctors diagnosing illness found little relationship between thor- 
oughness of data collection and accuracy of diagnosis. Medical 
students whose self-described research strategy stressed thorough col- 
lection of information (as opposed to formation and testing of hypoth- 
eses) were significantly below average in the accuracy of their 
diagnoses. It seems that the explicit formulation of hypotheses directs 
a more efficient and effective search for information.” 


Modeling Expert Judgment 


Another significant question concerns the extent to which ana- 
lysts possess an accurate understanding of their own mental processes. 
How good is their insight into how they actually weight evidence in 
making judgments? For each situation to be analyzed, they have an 
implicit “mental model” consisting of beliefs and assumptions as to 
which variables are most important and how they are related to each 
other. If analysts have good insight into their own mental model, they 
should be able to identify and describe the variables they have consid- 
ered most important in making judgments. 

There is strong experimental evidence, however, that such self- 
insight is usually faulty. The expert perceives his or her own judgmen- 
tal process, including the number of different kinds of information 


‘ For a list of references, see Lewis R. Goldberg. “Simple Models or Simple Processes’ Some 
Research on Clinical Judgments.” American Psychologist, 23 (1968), pp. 261-265 

Stuart Oskamp, “Overcontidence in Case-Study Judgments.” Journal of Consulting Psychologs. 
29 (1965), pp. 261-265 
* Arthur S. Elstemn ef al. Medical Problem Solving: An Aaalvysis of Clinical Reasoning (Cam 
bridge. MA and London: Harvard University Press, 1978), pp. 270 and 295 
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taken into account, as being considerably more complex than is in fact 
the case. Experts overestimate the importance of factors that have only 
a minor impact on their judgment and underestimate the extent to 
which their decisions are based on a few major variables. In short, 
people's mental models are simpler than they think, and the analyst ts 
typically unaware not only of which variables should have the greatest 
influence, but also which variables actually are having the greatest 
influence. 

All this has been demonstrated by experiments in which ana- 
lysts were asked to make quantitative estimates concerning a rela- 
tively large number of cases in their area of expertise, with each case 
defined by a number of quantifiable factors. In one experiment, for 
example, stock market analysts were asked to predict long-term price 
appreciation for 50 securities, with each security being described in 
such terms as price/earnings ratio, corporate earnings growth trend, 
and dividend yield.” After completing this task, the analysts were 
asked to explain how they reached their conclusions, including how 
much weight they attached to each of the variables hey were 
instructed to be sufficiently explicit that another person going through 
the same information could apply the same judgmental rules and 
arrive at the same conclusions. 

In order to compare this verbal rationalization with the judgmen- 
tal policy reflected in the stock market analysts’ actual decisions, mul- 
tiple regression analysis or other similar statistical procedures can be 
used to develop a mathematical model of how each analyst actuaily 
weighed and combined information on the relevant variables.” There 
have been at least eight studies of this type in diverse fields,” includ- 
ing one involving prediction of future socioeconomic growth of 
underdeveloped nations.“ The mathematical model based on the ana- 
lyst’s actual decisions is invariably a more accurate description of that 
analyst's decisionmaking than the analyst's own verbal description of 
how the judgments were made. 


Paul Slovic, Dan Fleissner, and W. Scott Bauman, “Analyzing the Use of Information in Invest 
ment Decision Making: A Methodological Proposal.” The Journal of Business, 45 (1972), pp. 284 
wl 
’ For a discussion of the methodology, see Slovic, Flenssner, and Bauman, op. cit 

For a list of references, see Paul Slovic and Sarah Lichtenstem, “Comparison of Bayesian and 
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von of Judgment Policy.” Prvchonomic Science, 18 (1970) pp. 249-250 


Although the existence of this phenomenon has been amply 
demonstrated, its causes are not well understood. The literature on 
these experiments contains only the following speculative explanation: 


Possibly our feeling that we can take into account a host of dif- 
ferent factors comes about because, although we remember that 
at some time or other we have attended to each of the different 
factors, we fail to notice that it is seldom more than one or two 
that we consider at any one time.” 


When Does New Information Affect Our Judgment? 

To evaluate the relevance and significance of these experimental 
findings in the context of intelligence analysts’ experiences, it is nec- 
essary to distinguish four types of additional information that an ana- 
lyst might receive: 


¢ Additional detail about variables already included in the 

analysis: Much raw intelligence reporting falls into this cate- 
gory. One would not expect such supplementary information to 
affect the overall accuracy of the analyst's judgment, and it is 
readily understandable that further detail that is consistent with 
previous information increases the analyst's confidence. Anal- 
yses for which considerable depth of detail is available to sup- 
port the conclusions tend to be more persuasive to their authors 
as well as to their readers. 


Identification of additional variables: Information on addi- 
tional variables permits the analyst to take into account other 
factors that may affect the situation. This is the kind of addi- 
tional information used in the horserace handicapper exper- 
ment. Other experiments have employed some combination of 
additional variables and additional detail on the same variables. 
The finding that judgments are based on a few critical variables 
rather than on the entire spectrum of evidence helps to explain 
why information on additional variables does not normally 
improve predictive accuracy. Occasionally, in situations when 


~ R.N. Shepard, “On Subjectively Optimum Selection Among Multuattribute Alternatives.” in M 
W. Shelly, Il and G. L. Bryan, eds.. Maman Judgments and Optimality (New York: Wiley, 1964). 
p 166 


there are known gaps in an analyst's understanding, a single 
report concerning some new and previously unconsidered fac- 
tor—for example, an authoritative report on a policy decision or 
planned coup d’etat—will have a major impact on the analyst's 
judgment. Such a report would fall into one of the next two cat- 
egories of new information. 


Information concerning the value attributed to variables 
already included in the analysis: An example of such infor- 
mation would be the horserace handicapper learning that a 
horse he thought would carry 110 pounds will actually carry 
only 106. Current intelligence reporting tends to deal with this 
kind of information; for example, an analyst may learn that a 
dissident group is stronger than had been anticipated. New facts 
affect the accuracy of judgments when they deal with changes 
in variables that are critical to the estimates. Analysts’ confi- 
dence in judgments based on such information is influenced by 
their confidence in the accuracy of the information as well as by 
the amount of information. 


Information concerning which variables are most impor- 
tant and how they relate to each other: Knowledge and 
assumptions as to which variables are most important and how 
they are interrelated comprise the mental model that tells the 
analyst how to analyze the data received. Explicit investigation 
of such relationships is one factor that distinguishes systematic 
research from current intelligence reporting and raw intelli- 
gence. In the context of the horserace handicapper experiment, 
for example, handicappers had to select which variables to 
include in their analysis. Is weight carried by a horse more, or 
less, important than several other variables that affect a horse's 
performance? Any information that affects this judgment influ- 
ences how the handicapper analyzes the available data; that is, 
it affects his mental model. 


The accuracy of an analyst's judgment depends upon both the 
accuracy of our mental model (the fourth type of information dis- 
cussed above) and the accuracy of the values attributed i the key 
variables in the model (the third type of information discussed above). 
Additional detail on variables already in the analyst's mental model 


and information on other variables that do not in fact have a signifi- 
cant influence on our judgment (the first and second types of informa- 
tion) have a negligible impact on accuracy, but form the bulk of the 
raw material analysts work with. These kinds of information increase 
confidence because the conclusions seem to be supported by such a 
large body of data. 

This discussion of types of new information is the basis for dis- 
tinguishing two types of analysis— data-driven analysis and conceptu- 
ally-driven analysis. 


Data-Driven Analysis 

In this type of analysis, accuracy depends primarily upon the 
accuracy and completeness of the available data. If one makes the rea- 
sonable assumption that the analytical model is correct and the fur- 
ther assumption that the analyst properly applies this model to the 
data, then the accuracy of the analytical judgment depends entirely 
upon the accuracy and completeness of the data. 

Analyzing the combat readiness of a military division is an 
example of data-driven analysis. In analyzing combat readiness, the 
rules and procedures to be followed are relatively well established. 
The totality of these procedures comprises a mental modei that influ- 
ences perception of the intelligence collected on the unit and guides 
judgment concerning what information is important and how this 
information should be analyzed to arrive at judgments concerning 
readiness. 

Most elements of the mental model can be made explicit so that 
other analysts may be taught to understand and follow the same ana- 
lytical procedures and arrive at the same or similar results. There ts 
broad, though not necessarily universal, agreement on what the appro- 
priate model is. There are relatively objective standards for judging 
the quality of analysis, inasmuch as the conclusions follow logically 
from the appli aison of the agreed-upon model to the available data. 


Conceptually Driven Analysis 

Conceptually driven analysis is at the opposite end of the spec- 
trum from data-driven analysis. The questions to be answered do not 
have neat boundaries, and there are many unknowns. The number of 
potentially relevant variables and the diverse and imperfectly under- 
stood relationships among these variables involve the analyst in enor- 
mous complexity and uncertainty. There is little tested theory to 


inform the analyst concerning which of the myriad pieces of informa- 
tion are most important and how they should be combined to arrive at 
probabilistic judgments. 

In the absence of any agreed-upon analytical schema, analysts 
are left to their own devices. They interpret information with the aid 
of mental models that are largely implicit rather than explicit. 
Assumptions concerning political forces and processes in the subject 
country may not be apparent even to the analyst. Such models are not 
representative of an analytical consensus. Other analysts examining 
the same data may well reach different conclusions, or reach the same 
conclusions but for different reasons. This analysis is conceptually 
driven, because the outcome depends at least as much upon the con- 
ceptual framework employed to analyze the data as it does upon the 
PO Oe ee ae ee es a a a oe cor 

To illustrate further the distinction between data-driven and con- 
ceptually driven analysis, it is useful to consider the function of the 
analyst responsible for current intelligence, especially current politi- 
cal intelligence as distinct from longer term research. The daily rou- 
tine is driven by the incoming wire service news, embassy cables, and 
clandestine-source reporting from overseas that must be interpreted 
for dissemination to consumers throughout the Intelligence Commu- 
nity. Although current intelligence reporting is driven by incoming 
information, this is not what is meant by data-driven analysis. On the 
contrary, the current intelligence analyst’s task is often extremely con- 
cept-driven. The analyst must provide immediate interpretation of the 
latest, often unexpected events. Apart from his or her store of back- 
ground information, the analyst may have no data other (..an the ini- 
tial, usually incomplete report. Under these circumstances, 
interpretation is based upon an implicit mental model of how and why 
events normally transpire in the country for which the analyst is 
responsible. Accuracy of judgment depends almost exclusively upon 
accuracy of the mental model, for there is little other basis for judg- 
ment. 

It is necessary to consider how this mental model gets tested 
against reality, and how it can be changed to improve the accuracy of 
analytical judgment. Two things make it hard to change one’s mental 
model. The first is the nature of human perception and information- 
processing. The second is the difficulty, in many fields, of learning 
what truly is an accurate model. 


60 


Partly because of the nature of human perception and informa- 
tion-processing, beliefs of all types tend to resist change. This is espe- 
cially true of the implicit assumptions and supposedly self-evident 
truths that play an important role in forming mental models. Analysts 
are often surprised to learn that what are to them self-evident truths 
are by no means self-evident to others, or that self-evident truth at one 
poirt in time may be conimonly regarded as uninformed assumption 
10 years later. 

Information that is consistent with an existing mind-set is per- 
ceived and processed easily and reinforces existing beliefs. Because 
the mind strives instinctively for consistency, information that is 
inconsistent with an existing mental image tends to be overlooked, 
perceived in a distorted manner, or rationalized to fit existing assump- 

tions and beliefs.” — a er ae ae . ae 4s 

Learning to make better judgments through cxperience assumes 
systematic feedback on the accuracy of previous judgments and an 
ability to link the accuracy of a judgment with the particular configu- 
ration of variables that prompted an analyst to make that judgment. In 
practice, iniclligence analysts get little systematic feedback, and even 
when they learn that an event they had foreseen has actually occurred 
or failed to occ, they typically do not know for certain whether this 
happened for the reasons they had foreseen. Thus, an analyst's per- 
sonal experience may be a poor guide to revision of his or her mental 
mode.” 


Mosaic Theory of Analysis 


Understanding of the analytic process has been distorted by the 
mosaic metaphor commonly used to describe it. According to the 
mosaic theory of intelligence, small pieces of information are col- 
lected that, when put together like a mosaic or jigsaw puzzle, eventu- 
ally enable analysts to perceive a clear picture of reality. The analogy 
suggests that accurate estimates depend primarily upon having all the 


~ This refers, of course, to subconscious processes. No analyst will consciously distort information 
that does not fit his or her preconceived beliefs. Important aspects of the perception and processing 
of new information occur prior to and independently of any conscious direction, and the tendencies 
described here are largely the result of these subconscious or preconscious processes 

* A similar point has been made in rebutting the belief in the accumulated wisdom of the classroom 
teacher. “It is actually very difficult for teachers to proat from experience. They almost never learn 
about their long-term successes or failures, and their short-term effects are not easily traced to the 
practices from which they presumably arose.” B. F. Skinner, The Technology of Teaching (New 
York: Appleton-Century Crofts, 1968), pp. 112-113. 
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pieces, that is, upon accurate and relatively complete information. It is 
important to collect and store the small pieces of information, as these 
are the raw material from which the picture is made: one never knows 
when it will be possible for an astute analyst to fit a piece into the 
puzzle. Part of the rationale for large technical intelligence collection 
systems is rooted in this mosaic theory. 

Insights from cognitive psychology suggest that intelligence ana- 
lysts do not work this way and that the most difficult analytical tasks 
cannot be approached in this manner. Analysts commonly find pieces 
that appear to fit many different pictures. Instead of a picture emerg- 
ing from putting all the pieces together, analysts typically form a pic- 
ture first and then select the pieces to fit. Accurate estimates depend at 
least as much upon the mental model used in forming the picture as 

* > Supon the nuiMtbe¥ of pieves of thé ptizzlée thai have been Collected? ~ ~ 

A more accurate analogy for describing how intelligence analy- 
sis should work is medical diagnosis. The doctor observes indicators 
(symptoms) of what is happening, uses his or her specialized knowl- 
edge of how the body works to develop hypotheses that might explain 
these observations, conducts tests to collect additional information to 
evaluate the hypotheses, then makes a diagnosis. This medical anal- 
ogy focuses attention on the ability to identify and evaluate all plausi- 
ble hypotheses. Collection is focused narrowly on information that 
will help to discriminate the relative probability of alternate hypothe- 
SiS. 

To the extent that this medical analogy is the more appropriate 
guide to understanding the analytical process, there are implications 
for the allocation of limited intelligence resources. While analysis and 
collection are both important, the medical analogy attributes more 
value to analysis and less to collection than the mosaic metaphor. 


Conclusions 


To the leaders and managers of intelligence who seek an 
improved intelligence product, these findings offer a reminder that this 
goal can be achieved by improving analysis as well as collection. 
There appear to be inherent practical limits on how much can be 
gained by efforts to improve collection. By contrast, an open and fer- 
tile field exists for imaginative efforts to improve analysis. 


These efforts should focus on improving the mental models 
employed by analysts to interpret information and the analytical pro- 
cesses used to evaluate it. While this will be difficult to achieve, it is 
so critical to effective intelligence ana ysis that even small improve- 
ments could have !arge benefits. Specific recommendations are 
included the next three chapters and in Chapter 14, “Improving Intelli- 
gence Analysis.” 
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Chapter 6 
Keeping an Open Mind 


Minds are like parachutes. They only function when they are 
open. After reviewing how and why thinking gets channeled into men- 
tal ruts, this chapter looks at mental tools to help analysts keep an 
open mind, question assumptions, see different perspectives, develop 
new ideas, and recognize when it is time to change their minds. 

* “A hew idea is the beginning; hot the end, of the ‘creative pro- 
cess. It must jump over many hurdles before being embraced as an 
organizational product or solution. The organizational climate plays 
a crucial role in determining whether new ideas bubble to the surface 
or are suppressed. 
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Major intelligence failures are usually caused by failures of anal- 
ysis, not failures of collection. Relevant information is discounted, 
misinterpreted, ignored, rejected, or overlooked because it fails to fit a 
prevailing mental model or mind-set.“ The “signals” are lost in the 
“noise.” How can we ensure that analysts remain open to new expe- 
rience and recognize when long-held views or conventional wisdom 
need to be revised in response to a changing world? 


“ Christopher Brady, “Intelligence Failures: Plus Ca Change ...“Intelligence and National Secu- 
rity, Vol. 8, No. 4 (October 1993). N. Cigar, “Iraq's Strategic Mindset and the Gulf War: Blueprint 
for Defeat.” The Journal of Strategic Studies, Vol. 15, No. | (March 1992). J. J. Wirtz, The Tet 
Offensive: Intelligence Failure in War (New York, 1991). Ephraim Kam, Surprise Attack (Harvard 
University Press, 1988). Richard Betts, Surprise Attack: Lessons for Defense Planning (Brook- 
ings, 1982). Abraham Ben-Zvi, “The Study of Surprise Attacks,” British Journal of International 
Studies, Vol. 5 (1979). Iran: Evaluation of Intelligence Performance Prior to November 1978 
(Staff Report, Subcommittee on Evaluation, Permanent Select Committee on Intelligence, US 
House of Representatives, January 1979). Richard Betts, “Analysis, War and Decision: Why Intel- 
ligence Failures Are Inevitable,” World Politics, Vol. 31, No. | (October 1978). Richard W. Shry- 
ock, “The Intelligence Community Post-Mortem Program, 1973-1975.” Studies in Intelligence. 
Vol. 21, No. 1 (Fall 1977). Avi Schlaim, “Failures in National Intelligence Estimates: The Case of 
the Yom Kippur War.” World Politics, Vol. 28 (April 1976). Michael Handel, Perception, Decep- 
tion, and Surprise: The Case of the Yom Kippur War (Jerusalem: Leonard Davis Institute of Inter- 
national Relations, Jerusalem Paper No. 19, 1976). Klaus Knorr, “Failures in National Intelligence 
Estimates: The Case of the Cuban Missiles.” World Politics, Vol. 16 (1964). 

“ Roberta Wohistetter, Pearl Harbor: Warning and Decision (Stanford University Press, 1962). 
Roberta Wohlstetter, “Cuba and Pearl Harbor: Hindsight and Foresight.” Foreign Affairs, Vol. 43, 
No. 4 (July 1965). 


Beliefs, assumptions, concepts, and information retrieved from 
memory form a mind-set or mental model that guides perception and 
processing of new information. The nature of the intelligence busi- 
ness forces us to deal with issues at an early stage when hard informa- 
tion is incomplete. If there were no gaps in the information on an 
issue Or situation, and no ambiguity, it would not be an interesting 
intelligence problem. When information is lacking, analysts often 
have no choice but to lean heavily on prior beliefs and assumptions 
about how and why events normally transpire in a given country. 

A mind-set is neither good nor bad. It is unavoidable. It is, in 
essence, a distillation of all that analysts think they know about a sub- 
ject. It forms a lens through which they perceive the world, and once 
formed, it resists change. 


Understanding Mental Ruts 


Chapter 3 on memory suggested thinking of information in 
memory as somehow interconnected like a massive, multidimensional 
spider web. It is possible to connect any point within this web to any 
other point. When analysts connect the same points frequently, they 
form a path that makes it easier to take that route in the future. Once 
they start thinking along certain channels, they tend to continue think- 
ing the same way and the path may become a rut. The path seems like 
the obvious and natural way to go. Information and concepts located 
near that path are readily available, so the same images keep coming 
up. Information not located near that path is less likely to come to 
mind. 

Talking about breaking mind-sets, or creativity, or even just 
openness to new information is really talking about spinning new 
links and new paths through the web of memory. These are links 
among facts and concepts, or between schemata for organizing facts 
or concepts, that were not directly connected or only weakly con- 
nected before. 

New ideas result from the association of old elements in new 
combinations. Previously remote elements of thought suddenly 


become associated in a new and useful combination.” When the link- 
age 1s made, the light dawns. This ability to bring previously unrelated 
information and ideas together in meaningful ways is what marks the 
open-minded, imaginative, creative analyst. 

To illustrate how the mind works, consider my personal experi- 
ence with a kind of mental block familiar to all analysts—writer’s 
block. I often need to break a mental block when writing. Everything 
is going along fine until | come to one paragraph and get stuck. I write 
something down, know it is not quite right, but just cannot think of a 
better way to say it. However I| try to change the paragraph, it still 
comes out basically the same way. My thinking has become chan- 
neled, and | cannot break out of that particular thought pattern to write 
it differently. 

A commgn.respopse to this problem is to take a break, work on 
something different for a while, and come back to the difficult portion 
later. With the passage of time, the path becomes less pronounced and 
it becomes easier to make other connections. 

1 have found another solution. | force myself to talk about it out 
loud. I close the door to my office—I am embarrassed to have anyone 
hear me talking to myself—and then stand up and walk around and 
talk. I say, okay, “What is the point of this paragraph? What are you 
trying to communicate” I answer myself out loud as though talking 
to someone else. “The point I am trying to get across is that ....” and 
then it just comes. Saying it out loud breaks the block, and words start 
coming together in different ways. 

Recent research explains why this happens. Scientists have 
learned that written language and spoken language are processed in 
different parts of the brain.” They activate different neurons. 


Problem-Solving Exercise 

Before discussing how analysts can keep their minds open to 
new information, let us warm up to this topic with a brief exercise. 
Without lifting pencil from paper, draw no more than four straight 
lines that will cross through all nine dots in Figure 6." 


“S.A. Mednick, “The Associative Basis of the Creative Process.” Psychological Review, Vol. 69 
(1962), p. 221. 

" Jerry E. Bishop, “Stroke Patients Yield Clues to Brain's Ability to Create Language.” Wall Street 
Journal, Oct. 12, 1993, p.Al. 

“ The puzzle is from James L. Adams, Conceptual Blockbusting: A Guide to Better Ideas. Second 
Edition (New York: W. W. Norton, 1980), p. 23. 
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After trying to solve the puzzle on your own, refer to the end of 
this chapter for answers and further discussion. Then consider that 
intelligence analysis is too often limited by similar, unconscious, self- 
imposed constraints or “cages of the mind.” 

You do not need to be constrained by conventional wisdom. It is 
often wrong. You do not necessarily need to be constrained by exist- 
ing policies. They can sometimes be chae*" if you show a good rea- 
son for doing so. You do not necessarily need to be constrained by the 
specific analytical requirement you were given. The policymaker who 
originated the requirement may not have thought through his or her 
needs or the requirement may be somewhat »urbled as it passes down 
through several echelons to you to do the v »:k. You may have a better 
understanding than the policymaker of what he or she needs, or 
should have, or what is possible to do. You should not hesitate to go 
back up the chain of command with a suggestion for doing something 
a little different than what was asked for. 


Mental Tools 


People use various physical tools such as a hammer and saw to 
enhance their capacity to perform various physical tasks. People can 
also use simple mental tools to enhance their ability to perform mental 
tasks. These tools help overcome limitatious in human mental machin- 
ery for perception, memory, and inference. The next few sections of 
this chapter discuss mental tools for opening analysts’ minds to new 
ideas, while the next one (Chapier 7) deals with mental tools for struc- 
turing complex analytical problems. 


Questioning Assumptions 

It is a truism that analysts need to question their assumptions. 
Experience tells us that when analytical judgments turn out to be 
wrong, it Qsually was tot because the information was wrong. It- was 
because an analyst made one or more faulty assumptions that went 
unchallenged. The problem is that analysts cannot question every- 
thing, so where do they focus their attention? 

Sensitivity Analysis. One approach is to do an informal sensitiv- 
ity analysis. How sensitive is the ultimate judgment to changes tn any 
of the major variables or driving forces in the analysis? Those linch- 
pin assumptions that drive the analysis are the ones that need to be 
questioned. Analysts should ask themselves what could happen to 
make any of these assumptions out of date, and how ihey can know 
this has not already happened. They should try to disprove their 
assumptions rather than confirm them. If an analyst cannot think of 
anything that would cause a change of mind, his or her mind-set may 
be so deeply entrenched that the analyst cannot see the conflicting evi- 
dence. One advantage of the competing hypotheses approach dis- 
cussed in Chapter 8 is that it helps identify the linchpin assumptions 
that swing a conclusion in one direction or another. 

Identify Alternative Models. Analysts should try to identify 
alternative models, conceptual frameworks, or interpretations of the 
data by seeking out individuals who disagree with them rather than 
those who agree. Most people do not do that very often. It is much 
more comfortable to talk with people in one’s own office who share 
the same basic mind-set. There are a few things that can be done as a 
matter of policy, and that have been done in some offices in the past. 
to help overcome this tendency. 
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At least one Directorate of Intelligence component, for example, 
has had a peer review process in which none of the reviewers was 
from the branch that produced the report. The rationale for this was 
that an analyst's immediate colleagues and supervisor(s) are likely to 
share a common mind-set. Hence these are the individuals least likely 
to raise fundamental issues challenging the validity of the analysis. To 
avoid this mind-set problem, each research report was reviewed by a 
committee of three analysts from other branches handling other coun- 
tries or issues. None of them had specialized knowledge of the sub- 
ject. They were, however, highly accomplished analysts. Precisely 
because they had not been immersed in the issue in question, they 
were better able to identify hidden assumptions and other alterna- 
tives, and to judge whether the analysis adequately supported the con- 
clusions. 

Be Wary of Mirrer Images.-One kind. of assumption an analyst 
should always recognize and question is mirror-imaging—filling gaps 
in the analyst's own knowledge by assuming that the other side is 
likely to act in a certain way because that is how the US would act 
under similar circumstances. To say, “if | were a Russian intelligence 
officer ...” or “if | were running the Indian Government ...” is mirror- 
imaging. Analysts may have to do that when they do not know how 
the Russian intelligence officer or the Indian Government is really 
thinking. But mirror-imaging leads to dangerous assumptions, because 
people in other cultures do nor think the way we do. The frequent 
assumption that they do is what Adm. David Jeremiah, after reviewing 
the Intelligence Community failure to predict India’s nuclear weap- 
ons testing, termed the “everybody-thinks-like-us mind-set.” 

Failure to understand that others perceive their national interests 
differently from the way we perceive those interests is a constant 
source of problems in intelligence analysis. In 1977, for example, the 
Intelligence Community was faced with evidence of what appeared to 
be a South African nuclear weapons test site. Many in the Intelli- 
gence Community, especially those least knowledgeable about South 
Africa, tended to dismiss this evidence on the grounds that “Pretoria 
would not want a nuclear weapon, because there is no enemy they 
could effectively use it on.”” The US perspective on what is in 
another country’s national interest is usually irrelevant in intelligence 


“ Jim Wolf, “CLA Inquest Finds US Missed Indian ‘Mindset’ UPI wire service, June 3, 1998 
Discussion with Robeit Jaster, former National Intelligence Officer for Southern Africa 
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analysis. Judgment must be based on how the other country perceives 
its national interest. If the analyst cannot gain insight into what the 
other country ts thinking, mirror-imaging may be the only alternative. 
but analysts should never get caught putting much confidence in that 
kind of judgment. 


Seeing Different Perspectives 

Another problem area is looking at familiar data from a differ- 
ent perspective. If you play chess, you know you can see your own 
options pretty well. It is much more difficult to see all the pieces on 
the board as your opponent sees them, and to anticipate how your 
opponent will react to your move. That is the situation analysts are in 
when they try to see how the US Government's actions look from 
another country’s perspective. Analysts constantly have to move back 
and forth, first seeing the situation from the US perspective and then 
from the other country’s perspective. This is difficult to do, as you 
experienced with the picture of the old woman/young woman in 
Chapter 2 on perception. 

Several techniques for seeing alternative perspectives exploit the 
general principle of coming at the problem from a different direction 
and asking different questions. These techniques break your existing 
mind-set by causing you to play a different and unaccustomed role. 

Thinking Backwards. One technique for exploring new ground 
is thinking backwards. As an intellectual exercise, start with an 
assumption that some event you did not expect has actually occurred. 
Then, put yourself into the future, looking back to explain how this 
could have happened. Think what must have happened six months or 
a year earlier to set the stage for that outcome, what must have hap- 
pened six months or a year before that to prepare the way, and so on 
back to the present. 

Thinking backwards changes the focus from whether something 
might happen to how it might happen. Putting yourself into the future 
creates a different perspective that keeps you from getting anchored in 
the present. Analysts will often find, to their surprise, that they can 
construct a quite plausible scenario for an event they had previously 
thought unlikely. Thinking backwards is particularly helpful for events 
that have a low probability but very serious consequences should they 
occur, such as a collapse or overthrow of the Saudi monarchy. 
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Crystal Ball. The crystal ball approach works in much the same 
way as thinking backwards.” Imagine that a “perfect” intelligence 
source (such as a crystal ball) has told you a certain assumption ts 
wrong. You must then develop a scenario to explain how this could be 
true. If you can develop a plausible scenario, this suggests your 
assuMption is Open to some question. 

Role playing. Role playing is commonly used to overcome con- 
straints and inhibitions that limit the range of one’s thinking. Playing a 
role changes “where you sit.” It also gives one license to think and act 
differently. Simply trying to iniagine how another leader or country 
will think and react, which analysts do frequently, is not role playing. 
One must actually act out the role and become, in a sense, the person 
whose role is assumed. It is only “living” the role that breaks an ana- 
lyst’s normal mental set and permits him or her to relate facts and 
ideas to each other in ways that differ from habitual patterns. An ana- 
lyst cannot be expected to do this alone; some group intéraction is 
required, with different analysts playing different roles, usually in the 
context of an organized simulation or game. 

Most of the gaming done in the Defense Department and in the 
academic world is rather elaborate and requires substantial prepara- 
tory work. It does not have to be that way. The preparatory work can 
be avoided by starting the game with the current situation already 
known to analysts, rather than with a notional scenario that partici- 
pants have to learn. Just one notional intelligence report is sufficient to 
start the action in the game. In my experience, it is possible to have a 
useful political game in just one day with almost no investment in pre- 
paratory work. 

Gaming gives no “right” answer, but it usually causes the players 
to see some things in a new light. Players become very conscious that 
“where you stand denends on where you sit.” By changing roles, the 
participants see the problem in a different context. This frees the mind 
to think differently. 

Devil's Advocate. A devil's advocate is someone who defends a 
minority point of view. He or she miy not necessarily agree with that 
view, but may choose or be assigned to represent it as strenuously as 
possible. The goal is to expose conflicting interpretations and show 


Jon Fallesen, Kex Michel, James Lussier, and Jutia Pounds, “Practical Thinking: Innovation in 
Battle Command Instruction” ( Technical Report 1087, US Army Research Institute for the Beha, 
ioral and Social Sciences, Jemuary 1996) 
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how alternative assumptions and images make the world look differ- 
ent. It often requires time, energy, and commitment to see how the 
world looks from a different perspective. ” 

Imagine that you are the boss at a US facility overseas and are 
worried about the possibility of a terrorist attack. A standard staff 
response would be to review existing measures and judge their ade- 
quacy. There might weil be pressure—subtle or otherwise—from 
those responsible for such arrangements to find them satisfactory. An 
alternative or supplementary approach would be to name an individual 
or smali group as a devil's advocate assigned to develop actual plans 
for launching such an attack. The assignment to think like a terrorist 
liberates the designated person(s) to think unconventionally and be 
less inhibited about finding weaknesses in the system that might 
embarrass colleagues, because uncovering any such weaknesses is the 
assigned task. 

Devil's advocacy has a controversial history in the Intelligence 
Community. Suffice it to say that some competition between conflict- 
ing views is healthy and must be encouraged; all-out political battle is 
counterproductive. 


Recognizing When To Change Your Mind 

As a genera! rule, people are too slow to change an established 
view, as opposed to being too willing to change. The human mind is 
conservative. It resists change. Assumptions that worked well in the 
past continue to be applied to new situations long after they have 
become outmoded. 

Learning from Surprise. A study of senior managers in indus- 
try identified hk ‘some successful managers counteract this conserva- 
tive bent. They do it, according to the study, 


By paying attention to their feelings of surprise when a particu- 
lar fact does not fit their prior understanding, and then by high- 
lighting rather than denying the novelty. Although surprise made 
them feel uncomfortable, it made them take the cause [of the sur- 
prise] seriously and inquire into it.... Rather than deny, downplay, 
or ignore disconfirm ‘ion [of their prior view], successful senior 


For an intere sting discussion of the strengths and potential weaknesses of the “devil's advocate” 
approach, see Robert Jervis, Perception and Misperception in International Politics (Princeton, NJ: 
Princeton University Press, 1976). pp. 415-418. 
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managers often treat it as friendly and in a way cherish the dis- 
comfort surprise creates. As a result, these managers often per- 
cei\. novel situations early on and in a frame of mind relatively 
undistorted by hidebound notions.” 


Analysts should keep a record of unexpected events and think 
hard about what they might mean, not disregard them or explain them 
away. It is important to consider whether these surprises, however 
small, are consistent with some alternative hypothesis. One unex- 
pected event may be easy to disregard, but a pattern of surprises may 
be the first clue that your understanding of whai 1s happening requires 
some adjustment, is at best incomplete, and may be quite wrong. 

Strategic Assumptions vs. Tactical Indicators. Abraham Ben- 
Zvi analyzed five cases of intelligence failure to foresee a surprise 
attack.’ He made a useful distinction between estimates based on stra- 
tegic assumptions and estimates based on tactical indications. Exam- 
ples of strategic assumptions include the US belief in 1941 that Japan 
wished to avoid war at all costs because it recognized US military 
superiority, and the Israeli belief in 1973 that the Arabs would not 
attack Israel until they obtained sufficient air power to secure control 
of the skies. A more recent instance was the 1998 Indian nuclear test, 
which was widely viewed as a surprise and, at least in part, as a fail- 
ure by the experts to warn of an impending test. The incorrect strate- 
gic assumption was that the new Indian Government would be 
dissuaded from testing nuclear weapons for fear of US economic 
sanctions.” 

Tactical indicators are specific reports of preparations or intent to 
initiate hostile action or, in the recent Indian case, reports of prepara- 
tions for a nuclear test. Ben-Zvi found that whenever strategic 
assumptions and tactical indicators of impending attack converged, an 
immediate threat was perceived and appropriate precautionary mea- 
sures were taken. When discrepancies existed between tactical indica- 
tors and strategic assumptions in the five cases Ben-Zvi analyzed, the 
strategic .ssumptions always prevailed, and they were never reevalu- 
ated in the light of the increasing flow of contradictory information. 


Daniel J. Isenberg. “How Senior Managers Think.” in David Bell, Howard Raiffa, and Amos 
Tversky, Decision Making: Descriptive, Normative, and Prescriptive Interactions (Cambridge 
University Press, 1988), p. 535. 

* Abraham Ben Zvi, “Hindsight and Foresight: A Conceptual Framework for the Analysis of Sur- 
prise Attacks,” World Politics, April 1976. 

* Transcript of Admiral David Jeremiah’s news conference on the Intelligence Community's per- 
formance concerning the Indian nuclear test, fourth and fifth paragraphs and first Q and A, 2 June 
1998. 


Ben-Zvi concludes that tactical indicators should be given increased 
weight in the decisionmaking process. At a minimum, the emergence 
of tactical indicators that contradict our strategic assumption should 
trigger a higher level of intelligence alert. It may indicate that a bigger 
surprise is on the way. 

Chapter 8, “Analysis of Competing Hypotheses,” provides a 
framework for identifying surprises and weighing tactical indicators 
and other forms of current evidence against longstanding assumptions 
and beliefs. 


Stimulating Creative Thinking 

Imagination and creativity play important roles in intelligence 
analysis as in most other human endeavors. Intelligence judgments 
require the ability to imagine possible causes and outcomes of a cur- 
rent situation. All possible outcomes are not given. The analyst must 
think of them by imagining scenarios (‘iat explicate how they might 
come about. Similarly, imagination as well as knowledge is required 
to reconstruct how a problem appears from the viewpoint of a foreign 
government. Creativity is required to question things that have long 
been taken for granted. The fact that apples fall from trees was well 
known to everyone. Newton’s creative genius was to ask “why?” 
Intelligence «.1alysts, too, are expected to raise new questions that lead 
to the identification of previously unrecognized relationships or to 
possible outcomes that had not previously been foreseen. 

A creative analytical product shows a flair for devising imagina- 
tive or innovative—but also accurate and effective—ways to fulfill any 
of the major requirements of analysis: gathering information, analyz- 
ing information, documenting evidence, and/or presenting conclu- 
sions. Tapping unusual sources of data, asking new questions, 
applying unusual analytic methods, and developing new types of 
prc ducts or new ways of fitting analysis to the needs o* consumers are 
all examples of creative activity. 

A person’s intelligence, as measured by IQ tests, has little to do 
with creativity, but the organizational environment exercises a major 
influence. New but appropriate ideas are most lil ly to arise in an 
organizational climate that nurtures their development and communi- 
cation. 

The old view that creativity is something one is born with, and 
that it cannot be taught or developed, is largely untrue. While native 
talent, per se, is important and may be immutable, it is possible to 
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learn to employ one’s innate talents more productively. With under- 
standing, practice, and conscious effort, analysts can learn to produce 
more imaginative, innovative, creative work. 

There is a large body of literature on creativity and how to stim- 
ulate it. At least a half-dozen different methods have been developed 
for teaching, facilitating, or liberating creative thinking. All the meth- 
ods for teaching or facilitating creativity are based on the assumption 
that the process of thinking can be separated from the content of 
thought. One learns mental strategies that can be applied to any sub- 
ject. 

It is not our purpose here to review commercially available pro- 
grams for enhancing creativity. Such programmatic approaches can be 
applied mure meaningfully to problems of new product development, 
advertising, or management than to intelligence analysis. It is relevant, 
however, to discuss several key principles and techniques that these 
programs have in common, and that individual intelligence analysts or 
groups of analysts can apply in their work. 

Intelligence analysts must generate ideas concerning potential 
causes or explanations of events, policies that might be pursued or 
actions taken by a foreign government, possible outcomes of an exist- 
ing situation, and variables that will influence which outcome actu- 
ally comes to pass. Analysts also need help to jog them out of mental 
ruts, to stimulate their memories and imaginations, and to perceive 
familiar events from a new perspective. 

Here are some of the principles and techniques of creative think- 
ing that can be applied to intelligence analysis. 

Deferred Judgment. The principle of deferred judgment is 
undoubtedly the most important. The idea-generation phase of analy- 
sis should be separated from the idea-evaluation phase, with evalua- 
tion deferrec until all possible ideas have been brought out. This 
approach runs contrary to the normal procedure of thinking of ideas 
and evaluating them concurrently. Stimulating the imagination and 
critical thinking are both important, but they do not mix well. A judg- 
mental attitude dampens the imagination, whether it manifests itself as 
self-censorship of one’s own ideas or fear of critical evaluation by col- 
leagues or supervisors. Idea generation should be a freewheeling, 
unconstrained, uncritical process. 

New ideas are, by definition, unconventional, and therefore 
likely to be suppressed, either consciously or unconsciously, unless 
they are born in a secure and protected environment. Critical judg- 
ment should be suspended until after the idea-generation stage of 
analysis has been completed. A series of ideas should be written down 
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and then evaluated later. This applies to idea searching by individuals 
as well as brainstorming in a group. Get all the ideas out on the table 
before evaluating any of them. 

Quantity Leads to Quality. A second principle is that quantity of 
ideas eventually leads to quality. This is based on the assumption that 
the first ideas that come to mind will be those that are most common 
or usual. It is necessary to run through these conventional ideas before 
arriving at original or different ones. People have habitual ways of 
thinking, ways that they continue to use because they have seemed 
successful in the past. It may well be that these habitual responses, the 
ones that come first to mind, are the best responses and that further 
search is unnecessary. In looking for usable new ideas, however, one 
should seek to generate as many ideas as possible before evaluating 
any of them. 

No Self-Imposed Constraints. A third principle is that thinking 
should be allowed—indeed encouraged—to range as freely as possi- 
ble. It is necessary to free oneself from self-imposed constraints, 
whether they stem from analytical habit, limited perspective, social 
norms, emotional blocks, or whatever. 

Cross-Fertilization of Ideas. A fourth principle of creative prob- 
lem-solving is that cross-fertilization of ideas is important and neces- 
sary. Ideas should be combined with each other to form more and 
even better ideas. If creative thinking involves forging new links 
between previously unrelated or weakly related concepts, then creativ- 
ity will be stimulated by any activity that brings more concepts into 
juxtaposition with each other in fresh ways. Interaction with other 
analysts is one basic mechanism for this. As a general rule, people 
generate more creative ideas when teamed up with others; they help to 
build and develop each other’s ideas. Personal interaction stimulates 
new associations between ideas. It also induces greater effort and 
helps maintain concentration on the task. 

These favorable comments on group processes are not meant to 
encompass standard committee meetings or coordination processes 
that force consensus based on the lowest common denominator of 
agreement. My positive words about group interaction apply prima- 
rily to brainstorming sessions aimed at generating new ideas and in 
which, according to the first principle discussed above, all criticism 
and evaluation are deferred until after the idea generation stage is 
completed. 

Thinking things out alone also has its advantages: individual 
thought tends to be more structured and systematic than interaction 
withi) a group. Optimal results come from alternating between 
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individual thinking and team effort, using group interaction to gen- 
erate ideas that supplement individual thought. A diverse group Is 
clearly preferable to a homogeneous one. Some group participants 
should be analysts who are not close to the problem, inasmuch as 
their ideas are more likely to reflect different insights. 

Idea Evaluation. All creativity techniques are concerned with 
stimulating the flow of ideas. There are no comparable techniques for 
determining which ideas are best. The procedures are, therefore, 
aimed at idea generation rather than idea evaluation. The same proce- 
dures do aid in evaluation, however, in the sense that ability to gener- 
ate more alternatives helps one see more potential consequences, 
repercussions, and effects that any single idea or action might entail. 


Organizational Environment 


A new idea is not the end product of the creative process. Rather, 
it is the beginning of what is sometimes a long and tortuous process of 
translating an idea into an innovative product. The idea must be devel- 
oped, evaluated, and communicated to others, and this process is 
influenced by the organizational setting in which it transpires. The 
potentially useful new idea must pass over a number of hurdles before 
it is embraced as an organizational product. 

The following paragraphs describe in some detail research con- 
ducted by Frank Andrews to investigate the relationship among cre- 
ative ability, organizational setting, and innovative research products.” 
The subjects of this research were 115 scientists, each of whom had 
directed a research project dealing with social-psychological aspects 
of disease. These scientists were given standardized tests that mea- 
sure creative ability and intelligence. They were also asked to fill out 
an extensive questionnaire concerning the environment in which their 
research was conducted. A panel of judges composed of the leading 
scientists in the field of medical sociology was asked to evaluate the 
principal published results from each of the 115 research projects. 

Judges evaluated the research results on the basis of productiv- 
ity and innovation. Productivity was defined as the “extent to which 
the research represents an addition to knowledge along established 
lines of research or as extensions of previous theory.” Innovativeness 
was defined as “additions to knowledge through new lines of research 


” Frank M. Andrews, “Social and Psychological Factors Which Influence the Creative Process,” 
in Irving A. Taylor and Jacob W. Getzels, eds., Perspectives in Creativity (Chicago, Aldine 
Publishing, 1975). 
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or the development of new theoretical statements of findings that were 
not explicit in previous theory.” Innovation, in other words, involved 
raising new questions and developing new approaches to. the acquisi- 
tion of knowledge, as distinct from working productively within an 
already established framework. This same definition applies to inno- 
vation in intelligence analysis. 

Andrews found virtually no relationship between the scientists’ 
creative ability and the innovativeness of their research. (There was 
also no relationship between level of intelligence and innovativeness.) 
Those who scored high on tests of creative ability did not necessarily 
receive high ratings from the judges evaluating the innovativeness of 
their work. A possible explanation is that either creative ability or 
innovation, or both, were not measured accurately, but Andrews 
argues persuasively for another view. Various social and psychological 
factors have so great an effect on the steps needed to translate cre- 
ative ability into an innovative research product that there is no mea- 
surable effect traceable to creative ability alone. In order to document 
this conclusion, Andrews analyzed data from the questionnaires in 
which the scientists described their work environment. 

Andrews found that scientists possessing more creative ability 
produced more innovative work only under the following favorable 
conditions: 


¢ When the scientist perceived himself or herself as responsible for 
initiating new activities. The opportunity for innovation, and the 
encouragement of it, are—not surprisingly—important variables. 


¢ When the scientist had considerable control over decisionmaking 
concerning his or her research program—in other words, the 
freedom to set goals, hire research assistants, and expend funds. 
Under these circumstances, a new idea is less likely to be snuffed 
out before it can be developed into a creative and useful product. 


¢ When the scientist felt secure and comfortable in his or her pro- 
fessional role. New ideas are often disruptive, and pursuing 
them carries the risk of failure. People are more likely to 
advance new ideas if they feel secure in their positions. 


” Ibid., p. 122. 
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¢ When the scientist’s administrative superior “stayed out of the 
way.” Research is likely to be more innovative when the supe- 
rior limits himself or herself to support and facilitation rather 
than direct involvement. 


¢ When the project was relatively small with respect to the num- 
ber of people involved, budget, and duration. Small size pro- 
motes flexibility, and this in turn is more conducive to 
creativity. 


¢ When the scientist engaged in other activities, such as teaching 
or administration, in addition to the research project. Other 
work may provide useful stimulation or help one identify 
opportunities for developing or implementing new ideas. Some 
time away from the task, or an incubation period, is generally 
recognized as part of the creative process. 


The importance of any one of these factors was not very great, 
but their impact was cumulative. The presence of all or most of these 
conditions exerted a strongly favorable influence on the creative pro- 
cess. Conversely, the absence of these conditions made it quite 
unlikely that even highly creative scientists could develop their new 
ideas into innovative research results. Under unfavorable conditions, 
the most creatively inclined scientists produced even less innovative 
work than their less imaginative colleagues, presumably because they 
experienced greater frustration with their work environment. 

In summary, some degree of innate creative talent may be a nec- 
essary precondition for innovative work, but it is unlikely to be of 
much value unless the organizational environment in which that work 
is done nurtures the development and communication of new ideas. 
Under unfavorable circumstances, an individual’s creative impulses 
probably will find expression outside the organization. 

There are, of course, exceptions to the rule. Some creativity 
occurs even in the face of intense opposition. A hostile environment 
can be stimulating, enlivening, and challenging. Some people gain sat- 
isfaction from viewing themselves as lonely fighters in the wilder- 
ness, but when it comes to conflict between a large organization and a 
Creative individual within it, the organization generally wins. 


Recognizing the role of organizational environment in stimulat- 
ing Or suppressing creativity points the way to one obvious set of 
measures to enhance creative organizational performance. Managers 
of analysis, from first-echelon supervisors to the Director of Central 
Intelligence, should take steps to strengthen and broaden the percep- 
tion among analysts that new ideas are welcome. This is not easy; cre- 
ativity implies criticism of that which already exists. It is, therefore, 
inherently disruptive of established ideas and organizational practices. 

Particularly within his or her own office, an analyst needs to 
enjoy a sense of security, so that partially developed ideas may be 
expressed and bounced off others as sounding boards with minimal 
fear of criticism or ridicule for deviating from established orthodoxy. 
At its inception, a new idea is frail and vulnerable. It needs to be nur- 
tured, developed, and tested in a protected environment before being 
exposed to the harsh reality of public criticism. It is the responsibility 
of an analyst’s immediate supervisor and office colleagues to provide 
this sheltered environment. 


Conclusions 


Creativity, in the sense of new and useful ideas, is at least as 
important in intelligence analysis as in any other human endeavor. 
Procedures to enhance innovative thinking are not new. Creative 
thinkers have employed them successfully for centuries. The only new 
elements—and even they may not be new anymore—are the ground- 
ing of these procedures in psychological theory to explain how and 
why they work, and their formalization in systematic creativity pro- 
grams. 

Learning creative problem-solving techniques does not change 
an analyst’s native-born talents but helps an analyst achieve his or her 
full potential. Most people have the ability to be more innovative than 
they themselves realize. The effectiveness of these procedures 
depends, in large measure, upon the analyst’s motivation, drive, and 
perseverance in taking the time required for thoughtful analysis 
despite the pressures of day-to-day duties, mail, and current 
intelligence reporting. 

A questioning attitude is a prerequisite to a successful search for 
new ideas. Any analyst who is confident that he or she already knows 
the answer, and that this answer has not changed recently, is unlikely 
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to produce innovative or imaginative work. Another prerequisite to 
creativity is sufficient strength of character to suggest new ideas to 
others, possibly at the expense of being rejected or even ridiculed on 
occasion. “The ideas of creative people often lead them into direct 
conflict with the trends of their time, and they need the courage to be 
able to stand alone.”” 


™ Robin Hogarth, Judgment and Choice (New York: Wiley, 1980), p. 117. 
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SOLUTIONS TO PUZZLE PRESENTED IN FIGURE 6 


Figure 6 


The nine-dots puzzle illustrated in Figure 6 above and earlier in this 
chapter is difficult to solve only if one defines the problem too nar- 
rowly. A surprising number of people assume they are not supposed to 
let the penci: go outside an imaginary square drawn around the nine dots. 


Figure 7 


This unconscious constraint exists only in the mind of the problem- 
solver; it is not specified in the definition of the problem. With no 
limit on the length of lines, it should be relatively easy to come up 
with the answer shown in Figure 7. 
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Another common, unconscious constraint is the assumption that the 
lines must pass through the center of the dots. This constraint, too, 
exists only in the mind of the problem solver. Without it, the three-line 
solution in Figure 8 becomes rather obvious. 


A more subtle and certainly more pervasive mental block is the 
assumption that such problems must be solved within a two-dimen- 
sional plane. By rolling the paper to form a cylinder, it becomes 
possible to draw a single straight line that spirals through all nine dots, 
as in Figure 9. 


Chapter 7 
Structuring Analytical Problems 


This ci.apter discusses various structures , * decomposing and 
externalizing complex analytical problems when we cannot keep all 
the relevant factors in the forefront of our consciousness at tiie same 
time. 

Decomposition means breaking a problem down into its compo- 
nent parts. Externalization means getting the problem out of our 
heads and into some visible form that we can work with. 

eeeeeee ee & ek Kk KK EK 


The discussion of working memory in Chapter 3 indicated that 
“The Magic Number Seven—Plus or Minus Two’’” is the number of 
things most people can keep in working memory at one time. To expe- 
rience firsthand this limitation on working memory while doing a 
mental task, try multiplying in your head any pair of two-digit num- 
bers— for example, 46 times 78. On paper, this ‘s a simple problem, 
but most people cannot keep track of that many numbers in their head. 

The limited capacity of working memory is the source of many 
problems in doing intelligence analysis. It is useful to consider just 
how complicated analysis can get, and how complexity might outstrip 
your working memory and impede your ability to make accurate judg- 
ments. Figure 10 illustrates how complexity increases geometrically 
as the number of variables in an analytical problem increases. The 
four-sided square shows that when a problem has just four variables, 
there are six possible interrelationships between those variables. With 
the pentagon, the five variables have 10 possible interrelationships. 
With six and eight variables, respectively, there are 15 and 28 possi- 
ble interrelationships between variables. 

The number of possible relationships between variables grows 
geometrically as the number of variables increases. 


* George A. Miller, “The Magical Number Seven, Plus or Minus Two: Some Limits on our Capac- 
ity for Processing Information.” The Ps) hological Review, Vol. 63, No. 2 (March 1956). 
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The number of possible relationships between variables 
grows geometrically as the number of variables increases. 


There are two basic tools for dealing with complexity in analy- 
sis—decomposition and externalization. 

Decomposition means breaking a problem down into its compo- 
nent parts. That is, indeed, the essence of analysis. Webster’s Dictio- 
nary defines analysis as division of a complex whole into its parts or 
elements.” 

The spirit of decision analysis is to divide and conquer: Decom- 
pose a complex problem into simpler problems, get one’s thinking 
straight in these simpler problems, paste these analyses together with 
a logical glue ...*! 

Externalization means getting the decomposed problem out of 
one’s head and down on paper or on a computer screen in some sim- 
plified form that shows the main variables, parameters, or elements of 


” Webster's Ninth New Collegiate Dictionary, 1988. 
"' Howard Raiffa, Decision Analysis (Reading, MA: Addison-Wesley, 1968). 
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the problem and how they relate to each other. Writing down the mul- 
tiplication problem, 46 times 78, is a very simple example of external- 
izing an analytical problem. When it is down on paper, one can easily 
manipulate one part of the problem at a time and often be more accu- 
rate than when trying to multiply the numbers in one’s head. 

I call this drawing a picture of your problem. Others call it mak- 
ing a model of your problem. It can be as simple as just making lists 
pro and con. 

This recommendation to compensate for limitations of working 
memory by decomposing and externalizing analytical problems is not 
new. The following quote is from a letter Benjamin Franklin wrote in 
1772 to the great British scientist Joseph Priestley, the discoverer of 
oxygen: 


In the affair of so much importance to you, wherein you ask my 
advice, | cannot for want of sufficient premises, advise you what 
to determine, but if you please I will tell you how. When those 
difficult cases occur, they are difficult, chiefly because while we 
have them under consideration, all the reasons pro and con are 
not present to the mind at the same time, but sometimes one set 
present themselves, and at other times another, the first being out 
of sight. Hence the various purposes or inclinations that alterna- 
tively prevail, and the uncertainty that perplexes us. 


To get over this, my way is to divide half a sheet of paper by a 
line into two columns; writing over the one Pro, and over the 
other Con. Then, during three or four days of consideration, I put 
down under the different heads short hints of the different 
motives, that at different times occur to me, for or against the 
measure. 


When I have thus got them all together in one view, I endeavor to 
estimate their respective weights; and where I find two, one on 
each side, that seem equal, I strike them both out. If I find a rea- 
son pro equal to some two reasons cca, I strike out the three ... 
and thus proceeding ! find at length where thc balance lies; and 
if, after a day or two of further consideration, nothing new that is 
of importance occurs on either side, I come to a determination 
accordingly. 
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And, though the weight of reasons cannot be taken with the pre- 
cision of algebraic quantities, yet when each is thus considered, 
separately and comparatively, and the whole lies before me, | 
think I can judge better, and am less liable to make a rash step, 
and in fact I have found great advantage from this kind of 
equation...." 


It is noteworthy that Franklin over 200 years ago identified the 
problem of limited working memory and how it affects one’s ability to 
make judgmc..». As Franklin noted, decision problems are difficult 
because people cannot keep all the pros and cons in mind at the same 
time. We focus first on one set of arguments and then on another, 
.., hence the various purposes and inclinations that alternatively pre- 
vail, and the uncertainty that perplexes us.” 

Franklin also identified the solution—getting all the pros and 
cons out of his head and onto paper in some visible, shorthand form. 
The fact that this topic was part of the dialogue between such illustri- 
ous individuals reflects the type of people who use such analytical 
tools. These are not aids to be used by weak analysts but unneeded by 
the strong. Basic limitations of working memory affect everyone. It is 
the more astute and careful analysts who are most conscious of this 
and most likely to recognize the value gained by applying these very 
simple tools. 

Putting ideas into visible form ensures that they will last. They 
will lie around for days goading you into having further thoughts. 
Lists are effective because they exploit people’s tendency to be a bit 
compulsive—we want to keep adding to them. They let us get the 
obvious and habitual answers out of the way, so that we can add to the 
list by thinking of other ideas beyond those that came first to mind. 
One specialist in creativity has observed that “for the purpose of mov- 
ing our minds, pencils can serve as crowbars”*'—just by writing 
things down and making lists that stimulate new associations. 

With the key elements of a problem written down in some abbre- 
viated form, it is far easier to work with each of the parts while still 
keeping the problem as a whole in view. Analysts can generally take 
account of more factors than when making a global judgment. They 
can manipulate individual elements of the problem to examine the 
many alternatives available through rearranging, combining, or modi- 
fying them. Variables may be given more weight or deleted, causal 
re.ctionships reconceptualized, or conceptual categories redefined. 


" J. Bigelow, ed., The Complete Works of Benjamin Franklin (New York: Putnam, 1887), p. 522. 
" Alex Osborn, Applied Imagination, Revised Edition (New York: Scribner's, 1979), p. 202. 


Such thoughts may arise spontaneously, but they are more likely to 
occur when an analyst looks at each element, one by one, and asks 
questions designed to encourage and facilitate consideration of alter- 
native interpretations. 


Problem Structure 


Anything that has parts also has a structure that relates these 
parts to each other. One of the first steps in doing analysis is to deter- 
mine an appropriate structure for the analytical problem, so that one 
can then identify the various parts and begin assembling information 
on them. Because there are many different kinds of analytical prob- 
lems, there are also many different ways to structure analysis. 

Lists such as Franklin made are one of the simplest structures. An 
intelligence analyst might make lists of relevant variables, early warn- 
ing indicators, alternative explanations, possible outcomes, factors a 
foreign leader will need to take into account when making a decision, 
or arguments for and against a given explanation or outcome. 

Other tools for structuring a problem include outlines, tables, 
diagrams, trees, and matrices, with many sub-species of each. For 
example, trees include decision trees and fault trees. Diagrams 
includes causal diagrams, influence diagrams, flow charts, and cogni- 
tive maps. 

Consideration of all those tools is beyond the scope of this book, 
but several such tools are discussed. Chapter 11, “Biases in Perception 
of Cause and Effect,” has a section on Illusory Correlation that uses a 
(2x2) contingency table to structure analysis of the question: Is decep- 
tion most likely when the stakes are very high? Chapter 8, “Analysis 
of Competing Hypotheses,” is arguably the most useful chapter in this 
book. It recommends using a matrix to array evidence for and against 
competing hypotheses to explain what is happening now or estimate 
what may happen in the future. 

The discussion below also uses a matrix to illustrate decomposi- 
tion and externalization and is intended to prepare you for the next 
chapter on “Analysis of Competing Hypotheses.” It demonstrates how 
to apply these tools to a type of decision commonly encountered in 
our personal lives. 
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Car Purchase Matrix 

In choosing among alternative purchases, such as when buying a 
car, a new computer, or a house, people often want to maximize their 
satisfaction on a number of sometimes-conflicting dimensions. They 
want a car at the lowest possible price, with the lowest maintenance 
cost, highest resale value, slickest styling, best handling, best gas 
mileage, largest trunk space, and so forth. They can’t have it all, so 
they must decide what is most important and make tradeoffs. As Ben 
Franklin said, the choice is sometimes difficult. We vacillate between 
one choice and another, because we cannot keep in working memory 
at the same time all the characteristics of all the choices. We think first 
of one and then the other. 

To handle this problem analytically, follow the divide-and-con- 
quer principle and “draw a picture” of the problem as a whole that 
helps you identify and make the tradeoffs. The component parts of the 
car purchase problem are the cars you are considering buying and the 
attributes or dimensions you want to maximize. After identifying the 
desirable attributes that will influence your decision, weigh how each 
car stacks up on each attribute. A matrix is the appropriate tool for 
keeping track of your judgments about each car and each attribute, 
and then putting all the parts back together to make a decision. 

Start by listing the important attributes you want to maximize, as 
shown for example in Figure 1! 1. 


Figure 11 


Next, quantify the relative importance of each attribute by divid- 
ing 100 percent among them. In other words, ask yourself what per- 
centage of the decision should be based on price, on styling, etc. This 
forces you to ask relevant questions and make decisions you might 
have glossed over if you had not broken the problem down in this 
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manner. How important is price versus styling, really? Do you really 
care what it looks like from the outside, or are you mainly looking for 
comfort on the inside and how it drives? Should safety be included in 
your list of important attributes? Because poor gas mileage can be off- 
set by lower maintenance cost for repairs, perhaps both should be 
combined into a single attribute called operating cost. 


Figure 12 


Safety 


Total 


This step might produce a result similar to Figure 12, depending 
on your personal preferences. If you do this together with your 
spouse, the exact basis of any difference of opinion will become 
immediately apparent and can be quantified. 

Next, identify the cars you are considering and judge how each 
one ranks on each of the six attributes shown in Figure 12. Set up a 
matrix as shown in Figure 13 and work across the rows of the matrix. 
For each attribute, take 10 points and divide it among the three cars 
based on how well they meet the requirements of that attribute. (This 
is the same as taking 100 percent and dividing it among the cars, but it 
keeps the numbers lower when you get to the next step.) 
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3.0% 


35% 


You now have a picture of your analytical problem—the com- 
parative value you attribute to each of the principal attributes of a new 
car and a comparison of how various cars satisfy those desired 
attributes. If you have narrowed it down to three alternatives, your 
matrix will look something like Figure 13: 

When all the cells of the matrix have been filled in, you can then 
calculate which car best suits your preferences. Multiply the percent- 
age value you assigned to each attribute by the value you assigned to 
that attribute for each car, which produces the result in Figure 14. If 
the percentage values you assigned to each attribute accurately reflect 
your preferences, and if each car has been analyzed accurately, the 
analysis shows you wil! gain more satisfaction from the purchase of 
Car 3 than either of the alternatives. 

At this point, you do a sensitivity analysis to determine whether 
plausible changes in some values in the matrix would swing the deci- 
sion to a different car. Assume, for example, that your spouse places 
different values than you on the relative importance of price versus 
styling. You can insert your spouse’s percentage values for those two 
attributes and see if that makes a difference in the decision. (For 


% Value Car 1 Car 2 Car 3 
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45 60 
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332.5 322.5 


example, one could reduce the importance of price to 20 percent and 
increase styling to 30 percent. That is still not quite enough to switch 
the choice to Car 2, which rates highest on styling.) 

There is a technical name for this type of analysis. It is called 
Multiattribute Utility Analysis, and there are complex computer pro- 
grams for doing it. In simplified form, however, it requires only pen- 
cil and paper and high school arithmetic. It is an appropriate structure 
for any purchase decision in which you must make tradeoffs between 
multiple competing preferences. 


Conclusions 


The car purchase example was a warmup for the following chap- 
ter. It illusuwates the difference between just sitting down and thinking 
about a problem and really analyzing a problem. The essence of analysis 
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is breaking down a problem into its component parts, assessing each part 
separately, then putting the parts back together to make a decision. The 
matrix in this example forms a “picture” of a complex problem by get- 
ting it out of our head and onto paper in a logical form that enables you 
to consider each of the parts individually. 

You certainly would not want to do this type of analysis for all 
your everyday personal decisions or for every intelligence judgment. 
You may wish to do it, however, for an especially important, difficult, 
or controversial judgment, or when you need to leave an audit trail 
showing how you arrived at.a judgment. The next chapter applies 
decomposition, externalization, and the matrix structure to a common 
type of intelligence problem. 


Chapter 8 


Analysis of Competing 
Hypotheses 


Analysis of competing hypotheses, sometimes abbreviated ACH, 
is a tool to aid judgment on important issues requiring careful weigh- 
ing of alternative explanations or conclusions. It helps an analyst 
overcome, or at least minimize, some of the cognitive limitations that 
make prescient intelligence analysis so difficult to achieve. 

ACH is an eight-step procedure grounded in basic insights from 
cognitive psychology, decision analysis, and the scientific method. It 
is a surprisingly effective, proven process that helps analysts avoid 
common analytic pitfalls. Because of its thoroughness, it is particu- 
larly appropriate for controversial issues when analysts want to leave 
an audit trail to show what they considered and how they arrived at 
their judgment.* 
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When working on difficult intelligence issues, analysts are, in 
effect, choosing among several alternative hypotheses. Which of sev- 
eral possible explanations is the correct one? Which of several possi- 
ble outcomes is the most likely one? As previously noted, this book 
uses the term “hypothesis” in its broadest sense as a potential explana- 
tion or conclusion that is to be tested by collecting and presenting evi- 
dence. 

Analysis of competing hypotheses (ACH) requires an analyst to 
explicitly identify all the reasonable alternatives and have them com- 
pete against each other for the analyst’s favor, rather than evaluating 
their plausibility one at a time. 

The way most analysts go about their business is to pick out 
what they suspect intuitively is the most likely answer, then look at 
the available information from the point of view of whether or not it 
supports this answer. If the evidence seems to support the favorite 


“ The analysis of competing hypotheses procedure was developed by the author for use by intelli- 
gence analysts dealing with a set of particularly difficult problems. 


hypothesis, analysts pat themselves on the back (“See, I knew it all 
along!”’) and look no further. If it does not, they either reject the evi- 
dence as misleading or develop another hypothesis and go through the 
same procedure again. Decision analysts call this a satisficing strat: zy. 
(See Chapter 4, Strategies for Analytical Judgment.) Satisficing means 
picking the first solution that seems satisfactory, rather than going 
through all the possibilities to identify the very best solution. There 
may be several seemingly satisfactory solutions, but there is only one 
best solution. 

Chapter 4 discussed the weaknesses in this approach. The princi- 
pal concern is that if analysts focus mainly on trying to confirm one 
hypothesis they think is probably true, they can easily be led astray by 
the fact that there is so much evidence to support their point of view. 
They fail to recognize that most of this evidence is also consistent 
with other explanations or conclusions, and that these other alterna- 
tives have not been refuted. 

Simultaneous evaluation of multiple, competing hypotheses is 
very difficult to do. To retain three to five or even seven hypotheses in 
working memory and note how each item of information fits into each 
hypothesis is beyond the mental capabilities of most people. It takes 
far greater mental agility than listing evidence supporting a single 
hypothesis that was pre-judged as the most likely answer. It can be 
accomplished, though, with the help of the simple procedures dis- 
cussed here. The box below contains a step-by-step outline of the 
ACH process. 


Step 1 


Identify the possible hypotheses to be considered. Use a 
group of analysts with different perspectives to brainstorm the 
possibilities. 

Psychological research into how people go about generating 
hypotheses shows that people are actually rather poor at thinking of 
all the possibilities.** If a person does not even generate the correct 
hypothesis for consideration, obviously he or she w''l not get the cor- 
rect answer. 


“ Charles Gettys et al., Hypothesis Generation: A Final Report on Three Years of Research, Tech- 
nical Report 15-10-80 (University of Oklahoma, Decision Processes Laboratory, 1980). 


Step-by-Step Outline of Analysis of Competing Hypotheses 


I. Identify the possible hypotheses to be considered. Use a group 
of analysts with different perspectives to brainstorm the possibili- 
ties. 


2. Make a list of significant evidence and arguments for and 
against each hypothesis. 


3. Prepare a matrix with hypotheses across the top and evidence 
down the side. Analyze the “diagnosticity” of the evidence and 
arguments—that is, identify which items are most helpful in judg- 
ing the relative likelihood of the hypotheses. 


4. Refine the matrix. Reconsider the hypotheses and delete evi- 
dence and arguments that have no diagnostic value. 


5. Draw tentative conclusions about the relative likelihood of 
each hypothesis. Proceed by trying to disprove the hyrotheses 
rather than prove them. 


6. Analyze how sensitive your conclusion is to a few critical 
items of evidence. Consider the consequences for your analysis if 
that evidence were wrong, misleading, or subject to a different 
interpretation. 


7. Report conclusions. Discuss the relative likelihood of all the 
hypotheses, not just the most likely one. 


8. Identify milestones for future observation that may indicate 
events are taking a different course than expected. 


It is useful to make a clear distinction between the hypothesis 


generation and hypothesis evaluation stages of analysis. Step | of the 
recommended analytical process is to identify all hypotheses that 


Jerit detailed examination. At this early hypothesis generation stage, 


it is very useful to bring together a group of analysts with different 
backgrounds and perspectives. Brainstorming in a group stimulates 
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the imagination and may bring out possibilities that individual mem- 
bers of the group had not thought of. Initial discussion in the group 
shculd elicit every possibility, no matter how remote, before judging 
likelihood or feasibility. Only when all the possibilities are on the 
table should you then focus on judging them and selecting the hypoth- 
eses to be examined in greater detail in subsequent analysis. 

When screening out the seemingly improbable hypotheses that 
you do not want to waste time on, it is necessary to distinguish 
hypotheses that appear to be disproved from those that are simply 
unproven. For an unproven hypothesis, there is no evidence that it is 
correct. For a disproved hypothesis, there is positive evidence that it is 
wrong. As discussed in Chapter 4, “Strategies for Analytical Judg- 
ment,” and under Step 5 below, you should seek evidence that dis- 
proves hypotheses. Ear'v rejection of unproven, but not disproved, 
hypotheses biases the subsequent analysis, because one does not then 
look for the evidence that might support them. Unproven hypotheses 
should be kept alive until they can be disproved. 

One example of a hypothesis that often falls into this unproven 
but not disproved category is the hypothesis that an opponent is try- 
ing to deceive us. You may reject the possibility of denial and decep- 
tion because you see no evidence of it, but rejection is not justified 
under these circumstances. If deception is planned well and properly 
implemented, one should not expect to find evidence of it readily at 
hand. The possibility should not be rejected until it is disproved, or, at 
least, until after a systematic search for evidence has been made and 
none has been found. 

There is no “correct” number of hypotheses to be considered. 
The number depends upon the nature of the analytical problem and 
how advanced you are in the analysis of it. As a general rule, the 
greater your level of uncertainty, or the greater the policy impact of 
your conclusion, the more alternatives you may wish to consider. 
More than seven hypotheses may be unmanageable; if there are this 
many alternatives, it may be advisable to group several of them 
together for your initial cut at the analysis. 
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Step 2 


Make a list of significant evidence and arguments for and 
against each hypothesis. 

In assembling the list of relevant evidence and arguments, these 
terms should be interpreted very broadly. They refer to all the factors 
that have an impact on your judgments aoout the hypotheses. Do not 
limit yourself to concrete evidence in the current intelligence report- 
ing. Also include your own assumptions or logical deductions about 
another person’s or group’s or country’s intentions, goals, or standard 
procedures. These assumptions may generate strong preconceptions as 
to which hypothesis is most like!y. Such assumptions often drive your 
final judgment, so it is important to include them in the list of “evi- 
dence.” 

First, list the general evidence that applies to all the hypotheses. 
Then consider each hypothesis individually, listing factors that tend to 
support or contradict each one. You will commonly find that each 
hypothesis leads you to ask different questions and, therefore, to seek 
out somewhat different evidence. 

For each hypothesis, ask yourself this question: If this hypothe- 
sis is true, what should I expect to be seeing or not seeing? What are 
all the things that must have happened, or may still be happening, and 
that one should expect to see evidence of? If you are not seeing this 
evidence, why not? Is it because it has not happened, it is not nor- 
mally observable, it is being concealed from you, or because you or 
the intelligence collectors have not looked for it? 

Note the at._.... of evidence as well as its presence. For exam- 
ple, when weighing the possibility of military attack by an adversary, 
the steps the adversary has not taken to ready his forces for attack 
may be more significant than the observable steps that have been 
taken. This recalls the Sherlock Holmes story in which the vital clue 
was that the dog did not bark in the night. One’s attention tends to 
focus on what is reported rather than what is not reported. It requires a 
conscious effort to think about what is missing but should be present 
if a given hypothesis were true. 


Step 3 


Prepare a matrix with hypotheses across the top and evi- 
dence down the side. Analyze the “diagnosticity” of the evidence 
and arguments- that is, identify which items are most helpful in 
judging the relative likelihood of alternative hypotheses. 

Step 3 is perhaps the most important element of this analytical 
procedure. It is also the step that differs most from the natural, intui- 
tive approach to analysis, and, therefore, the step you are most likely 
to overlook o: misunderstand. 

The procedure for Step 3 is to take the hypotheses from Step | 
and the evidence and arguments from Step 2 and put this information 
into a matrix format, with the hypotheses across the top and evidence 
and arguments down the side. This gives an overview of all the signif- 
icant components of your analytical problem. 

Then analyze how each piece of evidence relates to each hypoth- 
esis. This differs from the normal procedure, which is to look at one 
hypothesis at a time in order to consider how well the evidence sup- 
ports that hypothesis. That will be done later, in Step 5. At this point, 
in Step 3, take one item of evidence at a time, then consider how con- 
sistent that evidence is with each of the hypotheses. Here is how to 
remember this distinction. In Step 3, you work across the rows of the 
matrix, examining one item of evidence at a time to see how consis- 
tent that item of evidence is with each of the hypotheses. In Step 5, 
you work down the columns of the matrix, examining one hypothesis 
at a time, to see how consistent that hypothesis is with all the evi- 
dence. 

To fill in the matrix, take the first item of evidence and ask 
whether it is consistent with, inconsistent with, or irrelevant to each 
hypothesis. Then make a notation accordingly in the appropriate cell 
under each hypothesis in the matrix. The form of these notations in 
the matrix is a matter of personal preference. It may be pluses, 
minuses, and question marks. It may be C, I, and N/A standing for 
consistent, inconsistent, or not applicable. Or it may be some textual 
notation. In any event, it will be a simplification, a shorthand repre- 
sentation of the complex reasoning that went on as you thought about 
how the evidence relates to each hypothesis. 

After doing this for the first iim of evidence, then go on to the 
next item of evidence and repeat the process until all cells in the 
matrix are filled. Figure 15 shows an example of how such a matrix 
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might look. It uses as an example the intelligence question that arose 
after the US bombing of Iraqi intelligence headquarters in 1993: Will 
Iraq retaliate? The evidence in the matrix and how it is evaluated are 
hypothetical, fabricated for the purpose of providing a plausible exam- 
ple of the procedure. The matrix does not reflect actual evidence or 
judgments available at that time to the US Intelligence Community. 


Figure 15 
Question: Will Iraq Retaliate for US Bombing of Its Intelligence Headquarters? 


Hypoth« ses: 

H1 - frag will not retaliate. 

H2 - It will sponsor some minor terrorist actions. 

H3 - Iraq is planning a major terrorist attack, perhaps against one or more CIA 
installations. 


El. Saddam public statement of 
intent not to retaliate. 


E2. Absence of terrorist offensive 
during the 1991 Gulf War. 


E3. Assumption that Iraq would not 
want to provoke another US attack. 


E4. Increase in frequencyfength of 
monitored Iraqi agent radio broadcasts. 


ES. Iraqi embassies instructed to take 
increased security precautions. 


E6. Assumption that failure to retaliate would 
be unacceptable loss of face for Saddam. 


— 


The matrix format helps you weigh the diagnosticity of each 
item of evidence, which is a key difference between analysis of com- 
peting hypotheses and traditional analysis. Diagnosticity of evidence 
is an important concept that is, unfortunately, unfamiliar to many ana- 
lysts. It was introduced in Chapter 4, and that discussion is repeated 
here for your convenience. 

Diagnosticity may be illustrated by a medical analogy. A high- 
temperature reading may have great value in telling a doctor that a 
patient is sick, but relatively little value in determining which illness a 
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person is suffering from. Because a high temperature is consistent 
with so many possible hypotheses about a patient's illness, this evi- 
dence has limited diagnostic value in determining which illness 
(hypothesis) is the more likely one. 

Evidence is diagnostic when it influences your judgment on the 
relative likelihood of the various hypotheses identified in Step |. If an 
item of evidence seems consistent with all the hypotheses, it may have 
no diagnostic value. A common experience is to discover that most of 
the evidence supporting what you believe is the most likely i.ypothesis 
really is not very helpful, because that same evidence is also consis- 
tent with other hypotheses. When you do identify items that are highly 
diagnostic, these should drive your judgment. These are also the items 
for which you should re-check accuracy and consider alternative inter- 
pretations, as discussed in Step 6. 

In the hypothetical matrix dealing with Iraqi intentions, note that 
evidence designated “El” is assessed as consistent with all of the 
hypotheses. In ot! er words, it has no diagnostic value. This is because 
we did not give any credence to Saddam's public statement on this ques- 
tion. He might say he will not retaliate but then do so, or state that he 
will retaliate and then not do it. On the other hand, E4 is diagnostic: 
increased frequency or length of Iraqi agent radio broadcasts is more 
likely to be observed if the Iraqis are planning retaliation than if they are 
not. The double minus for E6 indicates this is considered a very strong 
argument against H1. It is a linchpin assumption that drives the conclu- 
sion in favor of either H2 or H3. Several of the judgments reflected in 
this matrix will be questioned at a later stage in this analysis. 

In some cases it may be useful to refine this procedure by using 
a numerical probability, rather than a general notation such as plus or 
minus, to describe how the evidence relates to each hypothesis. To do 
this, ask the following question for each cell in the matrix: If this 
hypothesis is true, what is the probability that | would be seeing this 
item of evidence? You may also make one or more additional nota- 
tions in each cell of the matrix, such as: 


¢ Adding a scale to show the intrinsic importance of each item of 
evidence. 


¢ Adding a scale to show the ease with which items of evidence 
could be concealed, manipulated, or faked, or the extent to 
which one party might have an incentive to do so. This may be 
appropriate when the possibility of denial and deception is a 
serious issue. 
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Step 4 


Refine the matrix. Reconsider the hypotheses and delete evi- 
dence and arguments that have no diagnostic value. 

The exact wording of the hypotheses is obviously critical to the 
conclusions one can draw from the analysis. By this point, you will 
have seen how the evidence breaks out under each hypothesis, and it 
will often be appropriate to reconsider and reword the hypotheses. Are 
there hypotheses that need to be added, or finer distinctions that need 
to be made in order to consider all the significant alternatives? If there 
is little or no evidence that helps distinguish between two hypotheses, 
should they be combined into one? 

Also reconsider the evidence. Is your thinking about which 
hypotheses are most likely and least likely influenced by factors that 
are not included in the listing of evidence? If so, put them in. Delete 
from the matrix items of evidence or assumptions that now seem 
unimportant or have no diagnostic value. Save these items in a sepa- 
rate list as a record of information that was considered. 


Step § 


Draw tentative conclusions about the relative likelihood of 
each hypothesis. Proceed by trying to disprove hypotheses rather 
than prove them. 

In Step 3, you worked across the matrix, focusing on a single 
item of evidence or argument and examining how it relates to each 
hypothesis. Now, work down the matrix, looking at each hypothesis as 
a whole. The matrix format gives an overview of all the evidence for 
and against all the hypotheses, so that you can examine all the hypoth- 
eses together and have them compete against each other for your 
favor. 

In evaluating the relative likelihood of alternative hypotheses, 
start by looking for evidence or logical deductions that enable you to 
reject hypotheses, or at least to determine that they are unlikely. A 
fundamental precept of the scientific method is to proceed by rejecting 
or eliminating hypotheses, while tentatively accepting only those 
hypotheses that cannot be refuted. The scientific method obviously 
cannot be applied in toto to intuitive judgment, but the principle of 
seeking to disprove hypotheses, rather than confirm them, is useful. 


No matter how much information is consistent with a given 
hypothesis, one cannot prove that hypothesis is true, because the same 
information may also be consistent with one or more other hypothe- 
ses. On the other hand, a single item of evidence that is inconsistent 
with a hypothesis may be sufficient grounds for rejecting that hypoth- 
esis. This was discussed in detail in Chapter 4, “Strategies for Analyti- 
cal Judgment.” 

People have a natural tendency to concentrate on confirming 
hypotheses they already believe to be true, and they commonly give 
more weight to information that supports a hypothesis than to infor- 
mation that weakens it. This is wrong; we should do just the opposite. 
Step 5 again requires doing the opposite of what comes naturally. 

In examining the matrix, look at the minuses, or whatever other 
notation you used to indicate evidence that may be inconsistent with a 
hypothesis. The hypotheses with the fewest minuses is probably the 
most likely one. The hypothesis with the most minuses is probably the 
least likely one. The fact that a hypothesis is inconsistent with the evi- 
dence is certainly a sound basis for rejecting it. The pluses, indicating 
evidence that is consistent with a hypothesis, are far less significant. It 
does not follow that the hypothesis with the most pluses is the most 
likely one, because a long list of evidence that is consistent with 
almost any reasonable hypothesis can be easily made. What is difficult 
to find, and is most significant when found, is hard evidence that is 
clearly inconsistent with a reasonable hypothesis. 

This initial ranking by number of minuses is only a rough rank- 
ing, however, as some evidence obviously is more important than 
other evidence, and degrees of inconsistency cannot be captured by a 
single notation such as a plus or minus. By reconsidering the exact 
nature of the relationship between the evidence and the hypotheses, 
you will be able to judge how much weight to give it. 

Analysts who follow this procedure often realize that their judg- 
ments are actually based on very few factors rather than on the large 
mass of information they thought was influencing their views. Chapter 
5, “Do You Really Need More Information?,” makes this same point 
based on experimental evidence. 

The matrix should not dictate the conclusion to you. Rather, it 
should accurately reflect your judgment of what is important and how 
these important factors relate to the probability of each hypothesis. 
You, not the matrix, must make the decision. The matrix serves only 
as an aid to thinking and analysis, to ensure consideration of all the 
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possible interrelationships between evidence and hypotheses and iden- 
tification of those few items that really swing your judgment on the 
issue. 

When the matrix shows that a given hypothesis is probable or 
unlikely, you may disagree. If so, it is because you omitted from the 
matrix One or more factors that have an important influence on your 
thinking. Go back and put them in, so that the analysis reflects your 
best judgment. If following this procedure has caused you to consider 
things you might otherwise have overlooked, or has caused you to 
revise your earlier estimate of the relative probabilities of the hypothe- 
ses, then the procedure has served a useful purpose. When you are 
done, the matrix serves as a shorthand record of your thinking and as 
an audit trail showing how you arrived at your conclusion. 

This procedure forces you to spend more analytical time than 
you otherwise would on what you had thought were the less likely 
hypotheses. This is desirable. The seemingly less likely hypotheses 
usually involve plowing new ground and, therefore, require more 
work. What you started out thinking was the most likely hypothesis 
tends to be based on a continuation of your own past thinking. A prin- 
cipal advantage of the analysis of competing hypotheses is that it 
forces you to give a fairer shake to all the alternatives. 


Step 6 


Analyze how sensitive your conclusion is to a few critical 
items of evidence. Consider the consequences for your analysis if 
that evidence were wrong, misleading, or subject to a different 
interpretation. 

In Step 3 you identified the evidence and arguments that were 
most diagnostic, and in Step 5 you used these findings to make tenta- 
tive judgments about the hypotheses. Now, go back and question the 
few linchpin assumptions or items of evidence that really drive the 
outcome of your analysis in one direction or the other. Are there ques- 
tionable assumptions that underlie your understanding and interpreta- 
tion? Are there alternative explanations or interpretations? Could the 
evidence be incomplete and, therefore, misleading? 

If there is any concern at all about denial and deception, this is 
an appropriate place io consider that possibility. Look at the sources 
of your key evidence. Are any of the sources known to the authorities 
in the foreign country? Could the information have been manipulated? 


Put yourself in the shoes of a foreign deception planner to evaluate 
motive, opportunity, means, costs, and benefits of deception as they 
might appear to the foreign country. 

When analysis turns out to be wrong, it is often because of key 
assumptions that went unchallenged and proved invalid. It is a truism 
that analysts should identify and question assumptions, but this ts 
much easier said than done. The problem is to determine which 
assumptions merit questioning. One advantage of the ACH procedure 
is that it tells you what needs to be rechecked. 

In Step 6 you may decide that additional research is needed to 
check key judgments. For example, it may be appropriate to go back 
to check original source materials rather than relying on someone 
else’s interpretation. In writing your report, it is desirable to identify 
critical assumptions that went into your interpretation and to note that 
your conclusion is dependent upon the validity of these assumptions. 


Step 7 


Report conclusions. Discuss the relative likelihood of all the 
hypotheses, not just the most likely one. 

If your report is to be used as the basis for decisionmaking, it 
will be helpful for the decisionmaker to know the relative likclihood 
of all the alternative possibilities. Analytical judgments are never cer- 
tain. There is always a good possibility of their being wrong. Deci- 
sionmakers need to make decisions on the basis of a full set of 
alternative possibilities, not just the single most likely alternative. 
Contingency or fallback plans may be needed in case one of the less 
likely alternatives turns out to be true. 

If you say that a certain hypothesis is probably true, that could 
mean anywhere from a 55-percent to an 85-percent chance that future 
events will prove it correct. That leaves anywhere from a 15-percent 
to 45 percent possibility that a decision based on your judgment will 
be based on faulty assumptions and will turn out wrong. Can you be 
more specific about how confident you are in your judgment? Chapter 
12, “Biases in Estimating Probabilities,” discusses the difference 
between such “subjective probability” judgments and statistical proba- 
bilities based on data on relative frequencies. 


When one recognizes the importance of proceeding by eliminat- 
ing rather than confirming hypotheses, it becomes apparent that any 
written argument for a certain judgment is incomplete unless it also 
discusses alternative judgments that were considered and why they 
were rejected. In the past, at least, this was seldom done. 

The narrative essay, which is the dominant art form for the pre- 
sentation of intelligence judgments, does not lend itself to compara- 
tive evaluation of competing hypotheses. Consideration of alternatives 
adds to the length of reports and is perceived by many analysts as 
detracting from the persuasiveness of argument for the judgment cho- 
sen. Analysts may fear that the reader could fasten on one of the 
rejected alternatives as a good idea. Discussion of alternative hypothe- 
ses is nonetheless an important part of any intelligence appraisal, and 
ways can and should be found to include it. 


Step 8 


Identify milestones for future observation that may indicate 
events are taking a different course than expected. 

Analytical conclusions should always be regarded as tentative. 
The situation may chan:e, or it may remain unchanged while you 
receive new ‘nformation that alters your appraisal. It is always helpful 
to specify in advance things one should look for or be alert to that, if 
observed, would suggest a significant change in the probabilities. This 
is useful for intelligence consumers who are following the situation on 
a continuing basis. Specifying in advance what would cause you to 
change your mind will also make it more difficult for you to rational- 
ize such developments, if they occur, as not really requiring any modi- 
fication of your judgment. 


Summary and Conclusion 


Three key elements distinguish analysis of competing hypothe- 
ses from conventional intuitive analysis. 


e Analysis starts with a full set of alternative possibilities, rather 
than with a most likely alternative for which the analyst seeks 
confirmation. This ensures that alternative hypotheses receive 
equal treatment and a fair shake. 


e Analysis identifies and emphasizes the few items of evidence or 
assumptions that have the greatest diagnostic value in judging 
the relative likelihood of the alternative hypotheses. In conven- 
tional intuitive analysis, the fact that key evidence may also be 
consistent with alternative hypotheses is rarely considered 
explicitly and often ignored. 


e Analysis of competing hypotheses involves seekir g evidence to 
refute hypotheses. The most probable hypothesis is usually the 
one with the least evidence against it, not the one » ‘th the most 
evidence for it. Conventional analysis generally entails looking 
for evidence to confirm a favored hypothesis. 


The analytical effectiveness of this procedure becomes apparent 
when considering the Indian nuclear weapons testing in 1998. 
According to Admiral Jeremiah, the Intelligence Community had 
reported that “ there was no indication the Indians would test in the 
near term.”” Such a conclusion by the Community would fail to d‘s- 
tinguish an unproven hypothesis from a disproved hypothesis. | .n 
absence of evidence does not necessarily disprove the hypothesis that 
India will indeed test nuclear weapons. 

If the ACH procedure had been used, one of the hypotheses 
would certainly have been that India is planning to test in the near 
term but will conceal preparations for the testing to forestall interna- 
tional pressure to halt such preparations. 

Careful consideration of this alternative hypothesis would have 
required evaluating India’s motive, opportunity, and means for con- 
cealing its intention until it was too late for the US and others to inter- 
vene. It would also have required assessing the ability of US 
intelligence to see through Indian denial and deception if it were 
being employed. It is hard to imagine that this would not have ele- 
vated awareness of the possibility of successful Iadian deception. 

A principal lesson is this. Whenever an intelligence analyst is 
tempted to write the phrase “there is no evidence that ...,” the analyst 
should ask this question: If this hypothesis is true, can I realistically 


“ Transcript of Adm. Jeremiah’s news conference, last sentence of third paragraph, 2 June 1998. 
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expect to see evidence of it? In other words, if India were planning 
nuclear tests while deliberately concealing its intentions, could the 
analyst realistically expect to see evidence of test planning? The ACH 
procedure leads the analyst to identify and face these kinds of ques- 
tions. 

Once you have gained practice in applying analysis of compet- 
ing hypotheses, it is quite possible to integrate the basic concepts of 
this procedure into your normal analytical thought process. In that 
case, the entire eight-step procedure may be unnecessary, except on 
highly controversial issues. 

There is no guarantee that ACH or any other procedure will pro- 
duce a correct answer. The result, after all, still depends on fallible 
intuitive judgment applied to incomplete and ambiguous information. 
Analysis of competing hypotheses does, however, ,uarantee an 
appropriate process of analysis. This procedure leaas you through 
a rational, systematic process that avoids some common analyti- 
cal pitfalls. It increases the odds of getting the right answer, and it 
leaves an audit trail showing the evidence used in your analysis 
and how this evidence was interpreted. If others disagree with 
your judgment, the matrix can be used to highlight the precise 
area of disagreement. Subsequent discussion can then focus pro- 
ductively on the ultimate source of the differences. 

A common experience is that analysis of competing hypotheses 
attributes greater likelihood to alternative hypotheses than would con- 
ventional analysis. One becomes less confident of what one thought 
ore knew. In focusing more attention on alternative explanations, the 
procedure brings out the full uncertainty inherent in any situation that 
is poor in data but rich in possibilities. Although such uncertainty is 
frusirating, it may oe an accurate reflection of the true situation. As 
Voltaire said, “Doubt is not a pleasant state, but certainty is a ridicu- 
lous one.’ 

The ACH procedure has the offsetting advantage of focusing 
attention on the few items of critical evidence that cause the uncer- 
tainty or which, if they were available, would alleviate it. This can 
guide future collection, research, and analysis to resolve the uncer- 
tainty and produce a more accurate judgment. 


“ M. Rogers, ed., Contradictory Quotations (England: Longman Group, Ltd., 1983). 
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PART THREE—COGNITIVE BIASES 
Chapter 9 
What Are Cognitive Biases? 


This mini-chapter discusses the nature of cognitive biases in 
general. The four chapters that follow it describe specific cognitive 
biases in the evaluation of evidence, perception of cause and effect, 
estimation of probabilities, and evaluation of intelligence reporting. 

eeeeee eee KEK KE & 


Fundamental limitations in human mental processes were identi- 
fied in Chapters 2 and 3. A substantial body of research in cognitive 
psychology and decisionmaking is based on the premise that these 
cognitive limitations cause people to employ various simplifying strat- 
egies and rules of thumb to ease the burden of mentally processing 
information to make judgments and decisions. These simple rules of 
thumb are often useful in helping us deal with complexity and ambi- 
guity. Under many circumstances, however, they lead to predictably 
faulty judgments known as cognitive biases. 

Cognitive biases are mental errors caused by our simplified 
information processing strategies. It is important to distinguish cogni- 
tive biases from other forms of bias, such as cultural bias, organiza- 
tional bias, or bias that results from one’s own self-interest. In other 
words, a cognitive bias does not result from any emotional or intellec- 
tual predisposition toward a certain judgment, but rather from subcon- 
scious mental procedures for processing information. A cognitive bias 
is a mental error that is consistent and predictable. For example: 


“ Much of this research was stimulated by the seminal work of Amos Tversky and Daniel Kahne- 
man, “Judgment under Uncertainty: Heuristics and Biases,” Science, 27 September 1974, Vol. 185, 
pp. 1124-1131. It has been summarized by Robin Hogarth, Judgement and Choice (New York: 
John Wiley & Sons, 1980), Richard Nisbett and Lee Ross, Human Inference: Strategies and Short- 
comings of Human Judgment (Englewood Cliffs, NJ: Prentice-Hall, 1980), and Robyn Dawes, 
Rational Choice in an Uncertain World (New York: Harcourt Brace Jovanovich College Publish- 
ers, 1988). The Hogarth book contains an excellent bibliography of research in this field, organized 
by subject. 
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The apparent distance of an object is determined in part by its 
clarity. The more sharply the object is seen, the closer it appears 
to be. This rule has some validity, because in any given scene the 
more distant objects are seen less sharply than nearer objects. 
However, the reliance on this rule leads to systematic errors in 
estimation of distance. Specifically, distances are often overesti- 
mated when visibility is poor because the contours of objects are 
blurred. On the other hand, distances are often underestimated 
when visibility is good because the objects are seen sharply. Thus 
the reliance on clarity as an indication of distance leads to com- 
mon biases.” 


This rule of thumb about judging distance is very useful. It usu- 
ally works and helps us deal with the ambiguity and complexity of life 
around us. Under certain predictable circumstances, however, it will 
lead to biased judgment. 

Cognitive biases are similar to optical illusions in that the error 
remains compelling even when one is fully aware of its nature. Aware- 
ness of the bias, by itself, does not produce a more accurate percep- 
tion. Cognitive biases, therefore, are, exceedingly difficult to 
overcome. 

Psychologists have conducted many experiments to identify the 
simplifying rules of thumb that people use to make judgments on 
incomplete or ambiguous information, and to show—at least in labo- 
ratory situations—how these rules of thumb prejudice judgments and 
decisions. The following four chapters discuss cognitive biases that 
are particularly pertinent to intelligence analysis because they affect 
the evaluation of evidence, perception of cause and effect, estimation 
of probabilities, and retrospective evaluation of intelligence reports. 

Before discussing the specific biases, it is appropriate to consider 
the nature of such experimental evidence and the extent to which one 
can generalize from these experiments to conclude that the same 
biases are prevalent in the Intelligence Community. 

When psychological experiments reveal the existence of a bias, 
this does not mean that every judgment by every individual person 
will be biased. It means that in any group of people, the bias will exist 
to a greater . 7 lesser degree in most judgments made by most of the 
group. On the basis of this kind of experimental evidence, one can 
only generalize about the tendencies of groups of people, not make 
statements about how any specific individual will think. 


*” Tversky and Kahneman, ibid. 
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I believe that conclusions based on these laboratory experiments 
can be generalized to apply to intelligence analysts. In most, although 
not all cases, the test subjects were experts in their field. They were 
physicians, stock market analysts, horserace handicappers, chess mas- 
ters, research directors, and professional psychologists, not undergrad- 
uate students as in so many psychological experiments. In most cases, 
the mental tasks performed in these experiments were realistic; that is, 
they were comparable to the judgments that specialists in these fields 
are normally required to make. 

Some margin for error always exists when extrapolating from 
experimental laboratory to real-world experience, but classes of CIA 
analysts to whom these ideas were presented found them relevant and 
enlightening. I replicated a number of the simpler experiments with 
military officers in the National Security Affairs Department of the 
Naval Postgraduate School. 
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Chapter 10 


Biases in Evaluation 
of Evidence 


Evaluation of evidence is a crucial step in analysis, but what 
evidence people rely on and how they interpret it are influenced by a 
variety of extraneous factors. Information presented in vivid and con- 
crete detail often has unwarranted impact, and people tend to disre- 
gard abstract or statistical information that may have greater 
evidential value. We seldom take the absence of evidence into 
account. The human mind is also oversensitive to the consistency of 
the evidence, and insufficiently sensitiv. to the reliability of the evi- 
dence. Finally, impressions often remain even after the evidence on 
which they are based has been totally discredited.” 
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The intelligence analyst works in a somewhat unique informa- 
tional environment. Evidence comes from an unusually diverse set of 
sources: newspapers and wire services, observations by American 
Embassy officers, reports from controlled agents and casual infor- 
mants, information exchanges with foreign governments, photo :econ- 
naissance, and communications intelligence. Each source has its own 
unique strengths, weaknesses, potential or actual biases, and vulnera- 
bility to manipulation and deception. The most salient characteristic 
of the information environment is its diversity—multiple sources, 
each with varying degrees of reliability, and each commonly report- 
ing information which by itself is incomplete and sometimes inconsis- 
tent or even incompatible -vith reporting from other sources. 
Conflicting information of uncertain reliability is endemic to intelli- 
gence analysis, as is the need to make rapid judgments on current 
events even before all the evidence is in. 


“ An earlier version of this chapter was published as an unclassified article in Studies in Intelli- 
gence in summer 1981, under the same title. 
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The analyst has only limited control over the stream of informa- 
tion. Tasking of sources to report on specific subjects is often a cum- 
bersome and time-consuming process. Evidence on some important 
topics is sporadic or nonexistent. Most human-source information is 
second hand at best. 

Recognizing and avoiding biases under such circumstances is 
particularly difficult. Most of the biases discussed in this chapter are 
unrelated to each other and are grouped together here only because 
they all concern some aspect of the evaluation of evidence. 


The Vividness Criterion 


The impact of information on the human mind is only imper- 
fectly related to its true value as evidence.” Specifically, information 
that is vivid, concrete, and personal has a greater impact on our think- 
ing than pallid, abstract information that may actually have substan- 
tially greater value as evidence. For example: 


¢ Information that people perceive directly, that they hear with 
their own ears or see with their own eyes, is likely to have 
greater impact than information received secondhand that may 
have greater evidential value. 


¢ Case histories and anecdotes will have greater impact than 
more informative but abstract aggregate or statistical data. 


Events that people experience personally are more memorable 
tliat those they only read about. Concrete words are easier to remem- 
ber than abstract words,” and words of all types are easier to recall 
than numbers. In short, information having the qualities cited in the 
preceding paragraph is more likely to attract and hold our attention. It 
is more likely to be stored and remembered than abstract reasoning or 
Statistical summaries, and therefore can be expected to have a greater 
immediate effect as well as a continuing impact on our thinking in the 
future. 


*' Most of the ideas and examples in this section are from Richard Nisbett and Lee Ross, Human 
Inference: Strategies and Shortcomings of Social Judgment (Englewood Cliffs, NJ: Prentice-Hall, 


1980), Chapter 3. 
*” A. Paivio, Imagery and Verbal Processes (New York: Holt, Rinehart & Winston, 1971). 
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Intelligence analysts generally work with secondhand informa- 
tion. The information that analysts receive is mediated by the written 
words of others rather than perceived directly with their own eyes and 
ears. Partly because of limitations imposed by their open CIA employ- 
ment, many intelligence analysts have spent less time in the country 
they are analyzing and had fewer contacts with nationals of that coun- 
try than their academic and other government colleagues. Occasions 
when an analyst does visit the country whose affairs he or she is ana- 
lyzing, or speaks directly with a national from that country, are mem- 
orable experiences. Such experiences are often a source of new 
insights, but they can also be deceptive. 

That concrete, sensory data do and should enjoy a certain prior- 
ity when weighing evidence is well established. When an abstract the- 
ory or secondhand report is contradicted by personal observation, the 
latter properly prevails under most circumstances. There are a num- 
ber of popular adages that advise mistrust of secondhand data: “Don’t 
believe everything you read,” “You can prove anything with statistics,” 
“Seeing is believing,” “I'm from Missouri...” 

It is curious that there are no comparable maxims to warn 
against being misled by our own observations. Seeing should not 
always be believing. 

Personal observations by intelligence analysts and agents can be 
as deceptive as secondhand accounts. Most individuals visiting for- 
eign countries become familiar with only a small sample of people 
representing a narrow segment of the total society. Incomplete and 
distorted perceptions are a common result. 

A familiar form of this error is the single, vivid case that out- 
weighs a much larger body of statistical evidence or conclusions 
reached by abstract reasoning. When a potential car buyer overhears a 
stranger complaining about how his Volvo turned out to be a lemon, 
this may have as much impact on the potential buyer’s thinking as sta- 
tistics in Consumer Reports on the average annual repair costs for for- 
eign-made cars. If the personal testimony comes from the potential 
buyer’s brother or close friend, it will probably be given even more 
weight. Yet the logical status of this new information is to increase by 
one the sample on which the Consumer Reports statistics were based; 
the personal experience of a single Volvo owner has little evidential 
value. 
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Nisbett and Ross label this the “man-who” syndrome and pro- 
vide the following illustrations:”' 


¢ “But I know a man who smoked three packs of cigarettes a day 
and lived to be ninety-nine.” 


e “I've never been to Turkey but just last month I met a man who 
had, and he found it...” 


Needless to say, a “man-who” example seldom merits the evi- 
dential weight intended by the person citing the example, or the 
weight often accorded to it by the recipient. 

The most serious implication of vividness as a criterion that 
determines the impact of evidence is that certain kinds of very valu- 
able evidence will have little influence simply because they are 
abstract. Statistical data, in particular, lack the rich and concrete detail 
to evoke vivid images, and they are often overlooked, ignored, or min- 
imized. 

For example, the Surgeon General’s report linking cigarette 
smoking to cancer should have, logically, caused a decline in per-cap- 
ita cigarette consumption. No such decline occurred for more than 20 
years. The reaction of physicians was particularly informative. All 
doctors were aware of the statistical evidence and were more exposed 
than the general population to the health problems caused by smok- 
ing. How they reacted to this evidence depended upon their medical 
specialty. Twenty years after the Surgeon General’s report, radiolo- 
gists who examine lung x-rays every day had the lowest rate of smok- 
ing. Physicians who diagnosed and treated lung canc.r victims were 
also quite unlikely to smoke. Many other types of physicians contin- 
ued to smoke. The probability that a physician continued to smoke 
was directly related to the distance of the physician’s specialty from 
the lungs. In other words, even physicians, who were well qualified to 
understand and appreciate the statistical data, were more influenced 
by their vivid personal experiences than by valid statistical data.“ 

Personal anecdotes, actual accounts of people’s responsiveness 
or indifference to information sources, and controlled experiments can 
all be cited ad infinitum “to illustrate the proposition that data summa- 
ries, despite their logically compelling implications, have less impact 


* Nisbett and Ross, p. 56. 
™ Ibid. 


than does inferior but more vivid evidence.” It seems likely that 
intelligence analysts, too, assign insufficient weight to statistical infor- 
mation. 

Analysts should give little weight to anecdotes and personal case 
histories unless they are known to be typical, and perhaps no weight at 
all if aggregate data based on a more valid sample can be obtained. 


Absence of Evidence 


A principal characteristic of intelligence analysis is that key 
information is often lacking. Analytical problems are selected on the 
basis of their importance and the perceived needs of the consumers, 
without much regard for availability of information. Analysts have to 
do the best they can with what they have, somehow taking into 
account the fact that much relevant information is known to be miss- 
ing. 

Ideally, intelligence analysts should be able to recognize what 
relevant evidence is lacking and factor this into their calculations. 
They should also be able to estimate the potential impact of the miss- 
ing data and to adjust confidence in their judgment accordingly. 
Unfortunately, this ideal does not appear to be the norm. Experiments 
suggest that “out of sight, out of mind” is a better description of the 
impact of gaps in the evidence. 

This problem has been demonstrated using fault trees, which are 
schematic drawings showing all the things that might go wrong with 
any endeavor. Fault trees are often used to study the fallibility of com- 
plex systems such as a nuclear reactor or space capsule. 

A fault tree showing all the reasons why a car might not start 
was shown to several groups of experienced mechanics.” The tree had 
seven major branches—insufficient battery charge, defective starting 
system, defective ignition system, defective fuel system, other engine 
problems, mischievous acts or vandalism, and all other problems— 
and a number of subcategories under each branch. One group was 
shown the full tree and asked to imagine 100 cases in which a car 
won't start. Members of this group were then asked to estimate how 
many of the 100 cases were attributable to each of the seven major 


* Nisbett and Ross, p. 57. 
“ Baruch Fischhoff, Paul Slovic, and Sarah Lichtenstein, Fault Trees: Sensitivity of Estimated Fail- 
ure Probabilities to Problem Representation, Technical Report PTR- | 042-77-8 (Eugene, OR: 
Decision Research, 1977). 


branches of the tree. A second group of mechanics was shown only an 
incomplete version of the tree: three major branches were omitted in 
order to test how sensitive the test subjects were to what was left out. 

If the mechanics’ judgment had been fully sensitive to the miss- 
ing information, then the number of cases of failure that would nor- 
mally be attributed to the omitted branches should have been added to 
the “Other Problems” category. In practice, however, the “Other Prob- 
lems” category was increased only half as much as it should have 
been. This indicated that the mechanics shown the incomplete tree 
were unable to fully recognize and incorporate into their judgments 
the fact that some of the causes for a car not starting were missing. 
When the same experiment was run with non-mechanics, the effect of 
the missing branches was much greater. 

As compared with most questions of intelligence analysis, the 
“car won't start” experiment involved rather simple analytical judg- 
ments based on information that was presented in a well-organized 
manner. That the presentation of relevant variables in the abbreviated 
fault tree was incomplete could and should have been recognized by 
the experienced mechanics selected as test subjects. Intelligence ana- 
lysts often have similar problems. Missing data is normal in intelli- 
gence problems, but it is probably more difficult to recognize that 
important information is absent and to incorporate this fact into judg- 
ments on intelligence questions than in the more concrete “car won't 
start” experiment. 

As an antidote for this problem, analysts should identify explic- 
itly those relevant variables on which information is lacking, consider 
alternative hypotheses concerning the status of these variables, and 
then modify their judgment and especially confidence in their judg- 
ment accordingly. They should also consider whether the absence of 
information is normal or is itself an indicator of unusual activity or 
inactivity. 


Oversensitivity to Consistency 


The internal consistency in a pattern of evidence helps determine 
our confidence in judgments based on that evidence.” In one sense, 
consistency is clearly an appropriate guideline for evaluating 
evidence. People formulate alternative explanations or estimates and 


* Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” 
Science, Vol. 185 (27 September 1974), 1126. 
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select the one that encompasses the greatest amount of evidence 
within a logically consistent scenario. Under some circumstances, 
however, consistency can be deceptive. Information may be consis- 
tent only because it is highly correlated or redundant, in which case 
many related reports may be no more informative than a single report. 
Or it may be consistent only because information is drawn from a 
very small sample or a biased sample. 

Such problems are most likely to arise in intelligence analysis 
when analysts have little information, say on political attitudes of 
Russian military officers or among certain African ethnic groups. If 
the available evidence is consistent, analysts will often overlook the 
fact that it represents a very small and hence unreliable sample taken 
from a large and heterogeneous group. This is not simply a matter of 
necessity—of having to work with the information on hand, however 
imperfect it may be. Rather, there is an illusion of validity caused by 
the consistency of the information. 

The tendency to place too much reliance on small samples has 
been dubbed the “law of small numbers.” This is a parody on the law 
of large numbers, the basic statistical principle that says very large 
samples will be highly representative of the population from which 
they are drawn. This is the principle that underlies opinion polling, but 
most people are not good intuitive statisticians. People do not have 
much intuitive feel for how large a sample has to be before they can 
draw valid conclusions from it. The so-called law of small numbers 
means that, intuitively, we make the mistake of treating small sam- 
ples as though they were large ones. 

This has been shown to be true even for mathematical psycholo- 
gists with extensive training in statistics. Psychologists designing 
experiments have seriously incorrect notions about the amount of 
error and unreliability inherent in small samples of data, unwarranted 
confidence in the early trends from the first few data points, and 
unreasonably high expectations of being able to repeat the same 
experiment and get the same results with a different set of test sub- 
jects. 

Are intelligence analysts also overly confident of conclusions 
drawn from very little data—especially if the data seem to be consis- 
tent? When working with a small but consistent body of evidence, 
analysts need to consider how representative that evidence is of the 


“ Tversky and Kahneman (1974), p. 1125-1126. 


total body of potentially available information. If more reporting were 
available, how likely is it that this information, too, would be consis- 
tent with the already available evidence? If an analyst is stuck with 
only a small amount of evidence and cannot deterniine how represen- 
tative this evidence is, confidence in judgments based on this evi- 
dence should be low regardless of the consistency of the information. 


Coping with Evidence of Uncertain Accuracy 


There are many reasons why information often is less than per- 
fectly accurate: misunderstanding, misperception, or having only part 
of the story; bias on the part of the ultimate source; distortion in the 
reporting chain from subsource through source, case officer, reports 
officer, to analyst; or misunderstanding and misperception by the ana- 
lyst. Further, much of the evidence analysts bring to bear in conduct- 
ing analysis is retrieved from memory, but analysts often cannot 
remember even the source of information they have in memory let 
alone the degree of certainty they attributed to the accuracy of that 
information when it was first received. 

The human mind has difficulty coping with complicated proba- 
bilistic relationships, so people tend to employ simple rules of thumb 
that reduce the burden of processing such information. In processing 
information of uncertain accuracy or reliability, analysts tend to make 
a simple yes or no decision. If they reject the evidence, they tend to 
reject it fully, so it plays no further role in their mental calculations. If 
they accept the evidence, they tend to accept it wholly, ignoring the 
probabilistic nature of the accuracy or reliability judgment. This is 
called a “best guess” strategy.” Such a strategy simplifies the integra- 
tion of probabilistic information, but at the expense of ignoring some 
of the uncertainty. If analysts have information about which they are 
70- or 80-percent certain but treat this information as though it were 
100-percent certain, judgments based on that information will be over- 
confident. 


” See Charles F. Gettys, Clinton W. Kelly III, and Cameron Peterson, “The Best Guess Hypothesis 
in Multistage Inference,” Organizational Behavior and Human Performance, 10, 3 (1973), 365- 
373, and David A. Schum and Wesley M. DuCharme, “Comments on the Relationship Between 
the Impact and the Reliability of Evidence,” Organizational Behavior and Human Performance, 6 
(1971), 111-131. 


122 


A more sophisticated strategy is to make a judgment based on an 
assumption that the available evidence is perfectly accurate and reli- 
able, then reduce the confidence in this judgment by a factor deter- 
mined by the assessed validity of the information. For example. 
available evidence may indicate that an event probably (75 percent) 
will occur, but the analyst cannot be certain that the evidence on 
which this judgment is based is wholly accurate or reliable. Therefore, 
the analyst reduces the assessed probability of the event (say, down to 
60 percent) to take into account the uncertainty concerning the evi- 
dence. This is an improvement over the best-guess strategy but gener- 
ally still results in judgments that are overconfident when compared 
with the mathematical formula for calculating probabilities.'” 

In mathematical terms, the joint probability of two events is 
equal to the product of their individual probabilities. Imagine a situa- 
tion in which you receive a report on event X that is probably (75 per- 
cent) true. If the report on event X is true, you judge that event Y will 
probably (75 percent) happen. The actual probability of Y is only 56 
percent, which is derived by multiplying 75 percent times 75 percent. 

In practice, life is not nearly so simple. Analysts must consider 
many items of evidence with different degrees of accuracy and reli- 
ability that are related in complex ways with varying degrees of prob- 
ability to several potential outcomes. Clearly, one cannot make neat 
mathematical calculations that take all of these probabilistic relation- 
ships into account. In making intuitive judgments, we unconsciously 
seek shortcuts for sorting through this maze, and these shortcuts 
involve some degree of ignoring the uncertainty inherent in less-than- 
perfectly-reliable information. There seems to be little an analyst can 
do about this, short of breaking the analytical problem down in a way 
that permits assigning probabilities to individual items of informa- 
tion, and then using a mathematical formula to integrate these separate 
probability judgments. 

The same processes may also affect our reaction to information 
that is plausib’e but known from the beginning to be of questionable 
authenticity. Ostensibly private statements by foreign officials are 
often reported though intelligence channels. In many instances it is 
not clear whether such a priv. 2 statement by a foreign ambassador, 
cabinet member, or other official is an actual statement of private 


Edgar M. Johnson, “The Effect of Data Source Reliability on Intuitive Inference,” Technical 
Paper 251 (Arlington, VA: US Army Research Institute for the Behavioral and Social Sciences, 
1974). 


views, an indiscretion, part of a deliberate attempt to deceive the US 
Government, or part of an apprcved plan to convey a truthful mes- 
sage that the foreign government believes is best transmitted through 
informal channels. 

The analyst who receives such a report often has little basis for 
judging the source’s motivation, so the information must be judged on 
its own merits. In making such an assessment, the analyst is influ- 
enced by plausible causal linkages. If these are linkages of which the 
analyst was already aware, the report has little impact inasmuch as it 
simply supports existing views. If there are plausible new linkages, 
however, thinking is restructured to take these into account. It seems 
likely that the impact on the analyst’s thinking is determined solely by 
the substance of the information, and that the caveat concerning the 
source does not attenuate the impact of the information at all. Know- 
ing that the information comes from an uncontrolled source who may 
be trying to manipulate us does not necessarily reduce the impact of 
the information. 


Persistence of Impressions Based on Discredited 
Evidence 


Impressions tend to persist even after the evidence that created 
those impressions has been fully discredited. Psychologists have 
become interested in this phenomenon because many of their experi- 
ments require that the test subjects be deceived. For example, test sub- 
jects may be made to believe they were successful or unsuccessful in 
performing some task, or that they possess certain abilities or person- 
ality traits, when this is not in fact the case. Professional ethics require 
that test subjects be disabused of these false impressions at the end of 
the experiment, but this has proved surprisingly difficult to achieve. 

Test subjects’ erroneous impressions concerning their logica! 
problem-solving abilities persevered even after they were informeu 
that manipulation of good or poor teaching performance had virtually 
guaranteed their success or failure.'"' Similarly, test subjects asked to 
distinguish true from fictitious suicide notes were given feedback that 


 R. R. Lau, M. R. Lepper, and L. Ross, “Persistence of Inaccurate and Discredited Personal 
Impressions: A Field Demonstration of Attributional Perseverance,” paper presented at S6th 
Annual Meeting of the Western Psychological Association (Los Angeles, April 1976). 


had no relationship to actual performance. The test subjects had been 
randomly divided into two groups, with members of one group being 
given the impression of above-average success and the other of rela- 
tive failure at this task. The subjects’ erroneous impressions of the dif- 
ficulty of the task and of their own performance persisted even after 
they were informed of the deception—that is, informed that their 
alleged performance had been preordained by their assignment to one 
or the other test group. Moreover, the same phenomenon was found 
among observers of the experiment as well as the immediate partici- 
pants. '” 

There are several cognitive processes that might account for this 
phenomenon. The tendency to interpret new information in the con- 
text of pre-existing impressions is relevant but probably not sufficient 
to explain why the pre-existing impression cannot be eradicated even 
when new information authoritatively discredits the evidence on 
which it is based. 

An interesting but speculative explanation is based on the strong 
tendency to seek causal explanations, as discussed in the next chapter. 
When evidence is first received, people postulate a set of causal con- 
nections that explains this evidence. In the experiment with suicide 
notes, for example, one test subject attributed her apparent success in 
distinguishing real from fictitious notes to her empathetic personality 
and the insights she gained from the writings of a novelist who com- 
mitted suicide. Another ascribed her apparent failure to lack of famil- 
larity with people who might contemplate suicide. The stronger the 
perceived causal linkage, the stronger the impression created by the 
evidence. 

Even after learning that the feedback concerning their perfor- 
mance was invalid, these subjects retained this plausible basis for 
inferring that they were either well or poorly qualified for the task. 
The previously perceived causal explanation of their ability or lack of 
ability still came easily to mind, independently of the now-discredited 
evidence that first brought it to mind.' Colloquially, one might say 
that once information rings a bell, the bell cannot be unrung. 


‘" Lee Ross, Mark R. Lepper, and Michael Hubbard, “Perseverance in Self-Perception and Social 
Perception: Biased Attributional Processes in the Debriefing Paradigm,” Journal of Personality and 
Social Psychology, 32, 5, (1975), 880-892. 

' Lee Res’. Mark R. Lepper, Fritz Strack, and Julia Steinmetz, “Social Explanation and Social 
Expectation: Effects of Real and Hypothetical Explanations on Subjective Likelihood,” Journal of 
Personality and Social Psychology, 33, \\ (1977), 818. 
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The ambiguity of most real-world situations contributes to the 
operation of this perseverance phenomenon. Rarely in the rez! world 
is evidence so thoroughly discredited as is possible in the experimen- 
tal laboratory. Imagine, for example, that you are told that a clandes- 
tine source who has been providing information for some time is 
actually under hostile control. Imagine further that you have formed a 
number of impressions on the basis of reporting from this source. It is 
easy to rationalize maintaining these impressions by arguing that the 
information was true despite the source being under control, or by 
doubting the validity of the report claiming the source to be under 
control. In the latter case, the perseverance of the impression may 
itself affect evaluation of the evidence that supposedly discredits the 
impression. 
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Chapter 11 


Biases in Perception of 
Cause and Effect 


Judgments about cause and effect are necessary to explain the 
past, understand the present, and estimate the future. These judgments 
are often biased by factors over which people exercise little con- 
scious control, and this can influence many types of judgments made 
by intelligence analysts. Because of a need to impose order on our 
environment, we seek and often believe we find causes for what are 
actually accidental or random phenomena. People overestimate the 
extent to which other countries are pursuing a coherent, coordinated, 
rational plan, and thus also overestimate their own ability to predict 
future events in those nations. People also tend to assume that causes 
are similar to their effects, in the sense that important or large effects 
must have large causes. 

When inferring the causes of behavior, too much weight is 
accorded to personal qualities and dispositions of the actor and not 
enough to situational determinants of the actor's behavior. People 
alsc overestimate their own importance as both « cause and a target 
of the behavior of others. Finally, people of . ceive relationships 
that do not in fact exist, because they do no; .. .¢ an intuitive under- 
standing of the kinds and amount of information needed to prove a 
relationship. 
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We cannot see cause and effect in the same sense that we see a 
desk or a tree. Even when we observe one billiard ball striking 
another and then watch the previously stationary ball begin to move, 
we are not perceiving cause and effect. The conclusion that one ball 
caused the other to move results only from a complex process of 
inference, not from direct sensory perception. That inference is based 
on the juxtaposition of events in time and space plus some theory or 
logical explanation as to why this happens. 


There are several modes of analysis by which one might infer 
cause and effect. In more formal analysis, inferences are made 
through procedures that collectively comprise the scientific method. 
The scientist advances a hypothesis, then tests this hypothesis by .he 
collection and statistical analysis of data on many instances of the 
phenomenon in question. Even then, causality cannot be proved 
beyond all possible doubt. The scientist seeks to disprove a hypothe- 
sis, not to confinn it. A hypothesis is accepted only when it cannot be 
rejected. 

Collection of data on many comparable cases to test hypotheses 
about cause and effect is not feasible for most questions of interest to 
the Intelligence Community, especially questions of broad political or 
strategic import relating to another country’s intentions. To be sure, it 
is feasible more often than it is done, and increased use of scientific 
procedures in political, economic, and strategic research is much to be 
encouraged. But the fact remains that the dominant approach to intel- 
lhgence analysis is necessarily quite different. It is the approach of the 
historian rather than the scientist, and this approach presents obsta- 
cles to accurate inferences about causality. 

The procedures and criteria most historians use to attribute cau- 
sality are less well defined than the scientist’s. 


The historian’s aim [is] to make a coherent whole out of the 
events he studies. His way of doing that, I suggest, is to look for 
Certain dominant concepts or leading ideas by which to illumi- 
nate his facts, to trace the connections between those ideas them- 
selves, and then to show how the detailed facts became 
intelligible in the light of them by constructing a “significant” 
narrative of the events of the period in question.'™ 


The key ideas here are coherence and narrative. These are the 
principles that guide the organization of observations into meaningful 
structures and patterns. The his orian commonly observes only a sin- 
gle case, not a pattern of covariation (when two things are related so 
that change in one is associated with change in the other) in many 
comparable cases. Moreover, the historian observes simultaneous 


'* 'W. H. Walsh, Philosophy of History: An Introduction (Revised Edition: New York: Harper and 
Row, 1967), p. 61. 
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changes in so many variables that the principle of covariation gener- 
ally is not helpful in sorting out the complex relationships among 
them. The narrative story, on the other hand, offers a means of orga- 
nizing the rich complexity of the historian’s observations. The histo- 
rian uses imagination to construct a coherent story out of fragments of 
data. 

The intelligence analyst employing the historical mode of analy- 
sis is essentially a storyteller. He or she constructs a plot from the pre- 
vious events, and this plot then dictates the possible endings of the 
incomplete story. The plot is formed of the “dominant concepts or 
leading ideas” that the analyst uses to postulate patterns of relation- 
ships among the available data. The analyst is not, of course, prepar- 
ing a work of fiction. There are constraints on the analyst's 
imagination, but imagination is nonetheless involved because there is 
an almost unlimited variety of ways in which the available data might 
be organized to tell a meaningful story. The constraints are the avail- 
able evidence and the principle of coherence. The story must form a 
logical and coherent whole and be internally consistent as well as con- 
sistent with the available evidence. 

Recognizing that the historical or narrative mode of analysis 
involves telling a coherent story helps explain the many disagreements 
among analysts, inasmuch as coherence is a subjective concept. It 
assumes some prior beliefs or mental model about what goes with 
what. More relevant to this discussion, the use of coherence rather than 
scientific observation as the criterion for judging truth leads to biases 
that presumably influence all analysts to some degree. Judgments of 
coherence may be influenced by many extraneous factors, and if ana- 
lysts tend to favor certain types of explanations as more coherent than 
others, they will be biased in favor of those explanations. 


Bias in Favor of Causal Explanations 


One bias attributable to the search for coherence is a tendency to 
favor causal explanations. Coherence implies order, so people natu- 
rally arrange observations into regular patterns and relationships. If no 
pattern is apparent, our first thought is that we lack understanding, not 
that we are dealing with random phenomena that have no purpose or 
reason. As a last resort, many people attribute happenings that they 
cannot understand to God’s will or to fate, which is somehow preor- 
dained; they resist the thought that outcomes may be determined by 
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forces that interact in random, unpredictable ways. People generally 
do not accept the notion of chance or randomness. Even dice players 
behave as though they exert some control over the outcome of a throw 
of dice." The prevalence of the word “because” in everyday lan- 
guage reflects the human tendency to seek to identify causes. 

People expect patterned events to look patterned, and random 
events to look random, but this is not the case. Random events often 
look patterned. The random process of flipping a coin six times may 
result in six consecutive heads. Of the 32 possible sequences result- 
ing from six coin flips, few actually look “random.”'” This is because 
randomness is a property of the process that generates the data that 
are produced. Randomness may in some cases be demonstrated by 
scientific (statistical) analysis. However, events will almost never be 
perceived intuitively as being random; one can find an apparent pat- 
tern in almost any set of data or create a coherent narrative from any 
set of events. 

Because of a need to impose order on their environment, people 
seek and often believe they find causes for what are actually random 
phenomena. During World War II, Londoners advanced a variety of 
causal explanations for the pattern of German bombing. Such explana- 
tions frequently guided their decisions about where to live and when 
to take refuge in air raid shelters. Postwar examination, however, 
determined that the clustering of bomb hits was close to a random dis- 
tribution." 

The Germans presumably intended a purposeful pattern, but pur- 
poses changed over time and they were not always achieved, so the 
net result was an almost random pattern of bomb hits. Londoners 
focused their attention on the few clusters of hits that supported their 
hypotheses concerning German inten'ions—not on the many cases 
that did not. 

Some research in paleobiology scems to illustrate the same ten- 
dency. A group of paleobiologists has developed a computer program 
to simulate evolutionary changes in animal species over time. But the 
transitions from one time period to the next are not determined by nat- 
ural selection or any other regular process: they are determined by 


'* Ellen J. Langer, “The Psychology of Chance,” Journal for the Theory of Social Behavior, 7 
(1977), 185-208. 

'* Daniel Kahneman and Amos Tversky, “Subjective Probability: A Judgment of Representative- 
ness,” Cognitive Psychology, 3 (1972), 430-54. 

'” W. Feller, An Introduction to Probability Theory and Its Applications (3rd Edition; New York: 
Wiley, 1968), p. 160. 
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computer-generated random numbers. The patterns produced by this 
program are similar to the patterns in nature that paleobiologists have 
been trying to understand. Hypothetical evolutionary events that seem, 
intuitively, to have a strong pattern were, in fact, generated by ran- 
dom processes. '" 

Yet another example of imposing causal explanations on ran- 
dom events is taken from a study dealing with the research practices 
of psychologists. When experimental results deviated from expecta- 
tions, these scientists rarely attributed the deviation to variance in the 
sample. They were always able to come up with a more persuasive 
causal explanation for the discrepancy.'” 

B. F. Skinner even noted a similar phenomenon in the course of 
experiments with the behavioral conditioning of pigeons. The normal 
pattern of these experiments was that the pigeons were given positive 
reinforcement, in the form of food, whenever they pecked on the 
proper lever at the proper time. To obtain the food regularly, they had 
to learn to peck in a certain sequence. Skinner demonstrated that the 
pigeons “learned” and followed a pattern (which Skinner termed a 
superstition) even when the food was uctually dispensed randomly.''’ 

These examples suggest that in military and foreign affairs, 
where the patterns are at best difficult to fathom, there may be many 
events for which there are no valid causal explanations. This certainly 
affects the predictability of events and suggests limitations on what 
might logically be expected of intelligence analysts. 


Bias Favoring Perception of Centralized Direction 


Very similar to the bias toward causal explanations is a tendency 
to see the actions of other governments (or groups of any type) as the 
intentional result of centralized direction and planning. “... most peo- 
ple are slow to perceive accidents, unintended consequences, coinci- 
dences, and small causes leading to large effects. Instead, coordinated 
actions, plans and conspiracies are seen.”''' Analysts overestimate the 


‘ Gina Bari Kolata, “Paleobiology: Random Events over Geological Time,” Science, 189 (1975), 
625-626. 

' Amos Tversky and Daniel Kahneman. “Belief in the Law of Small Numbers,” Psychological 
Bulletin, 72, 2 (1971), 105-110. 

'” B. F. Skinner, “Superstition in the Pigeon,” Journal of Experimental Psychology, 38 (1948), 168- 
172. 

'" Robert Jervis, Perception and Misperception in Iniernational Politics (Princeton, NJ: Princeton 
University Press, 1976), p. 320. 


extent to which other countries are pursuing coherent, rational, goal- 
maximizing policies, because this makes for more coherent, logical, 
rational explanations. This bias also leads analysts and policymakers 
alike to overestimate the predictability of future events in other coun- 
tries. 

Analysts know that outcomes are often caused by accident, blun- 
der, coincidence, the unintended consequence of well-intentioned pol- 
icy, improperly executed orders, bargaining among semi-independent 
bureaucratic entities, or following standard operating procedures 
under inappropriate circumstances.'’? But a focus on such causes 
implies a disorderly world in which outcomes are determined more by 
chance than purpose. It is especially difficult to incorporate these ran- 
dom and usually unpredictable elements into a coherent narrative, 
because evidence is seldom available to document them on a timely 
basis. It is only in historical perspective, after memoirs are written and 
government documents released, that the full story becomes available. 

This bias has important consequences. Assuming that a foreign 
government’s actions result from a logical and centrally directed plan 
leads an analyst to: 


e Have expectations regarding that government’s actions that 
may not be fulfilled if the behavior is actually the product of 
shifting or inconsistent values, bureaucratic bargaining, or 
sheer confusion and blunder. 


¢ Draw far-reaching but possibly unwarranted inferences from 
isolated statements or actions by government officials who may 
be acting on their own rather than on ceniral direction. 


¢ Overestimate the United States’ ability to influence the other 
government’s actions. 


¢ Perceive inconsistent policies as the result of duplicity and 
Machiavellian maneuvers, rather than as the product of weak 
leadership, vacillation, or bargaining among diverse bureau- 
cratic or political interests. 


'? For many historical examples, see Jervis, ibid., p. 321-23. 


Similarity of Cause and Effect 


When systematic analysis of covariation is not feasible and sev- 
eral alternative causal explanations seem possible, one rule of thumb 
people use to make judgments of cause and effect is to consider the 
similarity between attributes of the cause and attributes of the effect. 
Properties of the cause are “...inferred on the basis of being corre- 
spondent with or similar to properties of the effect.”''’ Heavy things 
make heavy noises; dainty things move daintily; large animals leave 
large tracks. When dealing with physical properties, such inferences 
are generally correct. 

People tend, however, to reason in the same way under circum- 
stances when this inference is not valid. Thus, analysts tend to assume 
that economic events have primarily economic causes, that big events 
have important consequences, and that little events cannot affect the 
course of history. Such correspcndence between cause and effect 
makes a more logical and persuasive—a more coherent—narrative, 
but there is little basis for expecting such inferences to correspond to 
historical fact. 

Fischer labels the assumption that a cause must somehow resem- 
ble its effect the “fallacy of identity,”''* and he cites as an example the 
historiography of the Spanish Armada. Over a period of several centu- 
ries, historians have written of the important consequences of the 
English defeat of the Spanish Armada in 1588. After refuting each of 
these arguments, Fischer notes: 


In short, it appears that the defeat of the Armada, mighty and 
melodramatic as it was, may have been remarkably barren of 
result. Its defeat may have caused very little, except the disrup- 
tion of the Spanish strategy that sent it on its way. That judg- 
ment is sure to violate the patriotic instincts of every Englishman 
and the aesthetic sensibilities of us all. A big event must have big 
results, we think.'' 


The tendency to reason according to similarity of cause and 
effect is frequently found in conjunction with the previously notea 
bias toward inferring centralized direction. Together, they explain the 
persuasiveness of conspiracy theories. Such theories are invoked to 
explain large effects for which there do not otherwise appear to be 
correspondingly large causes. For example, it seems “...outrageous 


Harold H. Kelley, “The Processes of Causal Attribution,” American Psychologist (February 
1973), p. 121. 

'* David Hackett Fischer, Historian's Fallacies (New York: Harper Torchbooks, 1970), p. 177. 

'S Ibid, p. 167. 
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that a single, pathetic, weak figure like Lee Harvey Oswald should 
alter world history.”''* Because the purported motive for the assassina- 
tion of John Kennedy is so dissimilar from the effect it is alleged to 
explain, in the minds of many it fails to meet the criterion of a coher- 
ent narrative explanation. If such “little” causes as mistakes, acci- 
dents, or the aberrant behavior of a single individual have big effects, 
then the implication follows that major events happen for reasons that 
are senseless and random rather than by purposeful direction. 

Intelligence analysts are more exposed than most people to hard 
evidence of real plots, coups, and conspiracies in the international 
arena. Despite this—or perhaps because of it—most intelligence ana- 
lysts are not especially prone to what are generally regarded as con- 
spiracy theories. Although analysts may not exhibit this bias in such 
extreme form, the bias presumably does influence analytical judg- 
ments in myriad little ways. In examining causal relationships, ana- 
lysts generally construct causal explanations that are somehow 
commensurate with the magnitude of their effects and that attribute 
events to human purposes or predictable forces rather than to human 
weakness, confusion, or unintended consequences. 


Internal vs. External Causes of Behavior 


Much research into how people assess the causes of behavior 
employs a basic dichotomy between internal determinants and exter- 
nal determinants of human actions. Internal causes of behavior 
include a person’s attitudes, beliefs, and personality. External causes 
include incentives and constraints, role requirements, social pres- 
sures, or other forces over which the individual has little control. The 
research examines the circumstances under which people attribute 
behavior either to stable dispositions of the actor or to characteristics 
of the situation to which the actor responds. 

Differences in judgments about what causes another person’s or 
government’s behavior affect how people respond to that behavior. 
How people respond to friendly or unfriendly actions by others may 
be quite different if they attribute the behavior to the nature of the per- 
son or government than if they see the behavior as resulting from situ- 
ational constraints over which the person or government has little 
control. 


'* Richard E. Nisiett and Timothy DeC. Wilson, “Telling More Than We Can Know: Verbal 
Reports on Mental Process...” Psychological Review (May 1977), p. 252. 
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A fundamental error made in judging the causes of behavior is to 
overestimate the role of internal factors and underestimate the role of 
external factors. When observing another’s behavior, people are too 
inclined to infer that the behavior was caused by broad personal quali- 
ties or dispositions of the other person and to expect that these same 
inherent qualities will determine the actor’s behavior under other cir- 
cumstances. Not enough weight is assigned to external circumstances 
that may have influenced the other person’s choice of behavior. This 
pervasive tendency has been demonsirated in many experiments under 
quite diverse circumstances''’’ and has often been observed in diplo- 
matic and military interactions.'" 

Susceptibility to this biased attribution of causality depends 
upon whether people are examining their own behavior or observing 
that of others. It is the behavior of others that people tend to attribute 
to the nature of the actor, whereas they see their own behavior as con- 
ditioned almost entirely by the situation in which they find them- 
selves. This difference is explained largely by differences in 
information available to actors and observers. People know a lot more 
about themselves. 

The actor has a detailed awareness of the history of his or her 
own actions under similar circumstances. In assessing the causes of 
our own behavior, we are likely to consider our previous behavior and 
focus on how it has been influenced by different situations. Thus situ- 
ational variables become the basis for explaining our own behavior. 
This contrasts with the observer, who typically lacks this detailed 
knowledge of the other person’s past behavior. The observer is 
inclined to focus on how the other person’s behavior compares with 
the behavior of others under similar circumstances.'” This difference 
in the type and amount of information available to actors and observ- 
ers applies to governments as well as people. 

An actor’s personal involvement with the actions being observed 
enhances the likelihood of bias. “Where the observer is also an actor, 
he is likely to exaggerate the uniqueness and emphasize the disposi- 
tional origins of the responses of others to his own actions.”'” This is 


'" Lee Ross, “The Intuitive Psychologist and his Shortcomings: Distortions in the Attribution Pro- 
cess,” in Leonard Berkowitz, ed., Advances in Experimental Social Psychology, Volume 10 (New 
York: Academic Press, 1977), p. 184. 

' Jervis, ibid., Chapter 2. 

''’ Edward E. Jones, “How Do People Perceive the Causes of Behavior?” American Scientist, 64 
(1976), p. 301. 

'® Daniel Heradstveit, The Arab-Israeli Conflict: Psychological Obstacles to Peace (Oslo: Univer- 
sitetsforlaget, 1979), p. 25. 
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because the observer assumes his or her own actions are unprovoca- 
tive, clearly understood by other actors, and well designed to elicit a 
desired response. Indeed, an observer interacting with another actor 
sees himself as determining the situation to which the other actor 
responds. When the actor does not respond as expected, the logical 
inference is that the response was caused by the nature of the actor 
rather than by the nature of the situation. 

Intelligence analysts are familiar with the problem of weighing 
internal versus external causes of behavior in a number of contexts. 
When a new leader assumes control of a foreign government, analysts 
assess the likely impact of changed leadership on government policy. 
For example, will the former Defense Minister who becomes Prime 
Minister continue to push for increases in the defense budget? Ana- 
lysts weigh the known predispositions of the new Prime Minister, 
based on performance in previous positions, against the requirements 
of the situation that constrain the available options. If relatively com- 
plete information is available on the situational constraints, analysts 
may make an accurate judgment on such questions. Lacking such 
information, they tend to err on the side of assuming that the individ- 
ual’s personal predispositions will prompt continuation of past behav- 
ior. 

Consider the Soviet invasion of Afghanistan. The Soviets’ per- 
ception of their own behavior was undoubtedly very different from the 
American perception. Causal attribution theory suggests that Soviet 
leaders would see the invasion as a reaction to the imperatives of the 
situation in South Asia at that time, such as the threat of Islamic 
nationalism spreading from Iran and Afghanistan into the Soviet 
Union. Further, they would perceive US failure to understand their 
“legitimate” national interests as caused by fundamental US hostil- 


ity. '?' 


'" See Richards J. Heuer, Jr, “Analyzing the Soviet Invasion of Afghanistan: Hypotheses from 
Causal Attribution Theory,” Studies in Comparative Communism, Winter 1980. These comments 
conceming the Soviet invasion of Afghanistan are based solely on the results of psychological 
research, not on information concerning Soviet actions in Afghanistan or the US reaction thereto. 
The nature of generalizations concerning how people normally process information is that they 
apply “more or less” to many cases but may not offer a perfect fit to any single instance. There were 
obviously many other factors that influenced analysis of Soviet actions, including preconceptions 
concerning the driving forces behind Soviet policy. The intent is to illustrate the relevance of psy- 
chological research on the analytical process, not to debate the merits of alternative interpretations 
of Soviet policy. Thus I leave to the reader to judge how much his or her own interpretation of the 
Soviet invasion of Afghanistan may be influenced by these attributional tendencies. 
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Conversely, observers of the Soviet invasion would be inclined 
to attribute it to the aggressive and expansionist nature of the Soviet 
regime. Dislike of the Soviet Union and lack of information on the sit- 
uational constraints as perceived by the Soviets themselves would be 
likely to exacerbate the attributional bias.'? Further, to the extent that 
this bias stemmed from insufficient knowledge of situational pressures 
and constraints, one might expect policymakers who were not Soviet 
experts to have had a stronger bias than analysts specializing in the 
Soviet Union. With their greater base of information on the situa- 
tional variables, the specialists may be better able to take these vari- 
ables into account. 

Specialists on occasion become so deeply immersed in the 
affairs of the country they are analyzing that they begin to assume the 
perspective—and the biases—of that country’s leaders. During the 
Cold War, there was a persistent difference between CIA specialists in 
Soviet affairs and specialists in Chinese affairs when dealing with 
Sino-Soviet relations. During border clashes in 1969, for example, 
specialists on the USSR argued that the Chinese were being “provoca- 
tive.” These specialists tended to accept the Soviet regime’s versions 
as to the history and alignment of the border. Specialists in Chinese 
affairs tended to take the opposite view—that is, that the arrogant 
Russians were behaving like Russians often do, while the Chinese 
were simply reacting to the Soviet high-handedness.'*’ In other words, 
the analysts assumed the same biased perspective as the leaders of the 
country about which they were most knowledgeable. An objective 
account of causal relationships might have been somewhere between 
these two positions. 

The Egypt-Israel peace negotiations in 1978-1979 offered 
another example of apparent bias in causal attribution. In the words of 
one observer at the time: 


Egyptians attribute their willingness to sign a treaty with Israel as 
due to their inherent disposition for peace; Israelis explain Egyp- 
tian willingness to make peace as resulting from a deteriorating 
economy and a growing awareness of Israel’s military superior- 
ity. On the other hand, Israelis attribute their own orientation for 
accommodation as being due to their ever-present preference for 


‘= Edward Jones and Richard Nisbett, “The Actor and the Observer: Divergent Perceptions of Their 
Behavior.” in Edward Jones et al., Attribution: Perceiving the Causes of Behavior (New Jersey: 
General Learning Press, 1971), p. 93. 

** Based on personal discussion with CIA analysts. 
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peace. Egypt, however, explains Israel’s compromises regarding, 
for example, Sinai, as resulting from external pressures such as 
positive inducements and threats of negative sanctions by the 
United States. In addition, some Egyptians attribute Israel’s 
undesirable behavior, such as establishment of Jewish settlements 
on the West Bank of the Jordan River, as stemming from Zionist 
expansionism. If Israel should not place settlements in that terri- 
tory, Egyptians might account for such desirable behavior as 
being due to external constraints, such as Western condemnation 
of settlements. Israelis, on the other hand explain undesirable 
behavior, such as Egypt’s past tendency to issue threats to drive 
them into the sea, as resulting from Egypt’s inherent opposition 
to a Jewish state in the Middle East. When Egyptians ceased to 
make such threats, Israelis attributed this desirable behavior as 
emanating from external circumstances, such as Israel’s relative 
military superiority. '* 


The persistent tendency to attribute cause and effect in this man- 
ner is not simply the consequence of self-interest or propaganda by 
the opposing sides. Rather, it is the readily understandable and pre- 
dictable result of how people normally attribute causality under many 
different circumstances. 

As a general rule, biased attribution of causality helps sow the 
seeds of mistrust and misunderstanding between people and between 
governments. We tend to have quite different perceptions of the 
causes of each other’s behavior. 


Overestimating Our Own Importance 


Individuals and governments tend to overestimate the extent to 
which they successfully influence the behavior of others.'* This is an 
exception to the previously noted generalization that observers 
attribute the behavior of others to the nature of the actor. It occurs 
largely because a person is so familiar with his or her own efforts to 
influence another, but much less well informed about other factors that 
may have influenced the other’s decision. 


'* Raymond Tanter, “Bounded Rationality and Decision Aids,” essay prepared for the Strategies of 
Conflict seminar, Mont Pelerin, Switzerland, 11-16 May 1980. 
'S This section draws heavily upon Jervis, Chapter 9. 
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In estimating the influence of US policy on the actions of 
another government, analysts more often than not will be knowledge- 
able of US actions and what they are intended to achieve, but in many 
instances they will be less well informed concerning the internal pro- 
cesses, political pressures, policy conflicts, and other influences on the 
decision of the target government. 

This bias may have played a role in the recent US failure to 
anticipate Indian nuclear weapons testing even though the new Indian 
Government was elected partly on promises it would add nuclear 
weapons to India’s military arsenal. Most US intelligence analysts 
apparently discounted the promises as campaign rhetoric, believing 
that India would be dissuaded from joining the nuclear club by eco- 
nomic sanctions and diplomatic pressure. Analysts overestimated the 
ability of US policy to influence Indian decisions. 

When another country’s actions are consistent with US desires, 
the most obvious explanation, in the absence of strong evidence to the 
contrary, is that US policy effectively influenced the decision.'** Con- 
versely, when another country behaves in an undesired manner, this is 
normally attributed to factors beyond US control. People and govern- 
ments seldom consider the possibility that their own actions have nad 
unintended consequences. They assume that their intentions have been 
correctly perceived and that actions will have the desired effect unless 
frustrated by external causes. 

Many surveys and laboratory experiments have shown that peo- 
ple generally perceive their own actions as the cause of their successes 
but not of their failures. When children or students or workers perform 
well, their parents, teachers, or supervisors take at least part of the 
credit; when they do poorly, their mentors seldom assume any blame. 
Successful candidates for Congress generally believe their own behav- 
ior contributed strongly to their victory, while unsuccessful candi- 
dates blame defeat on factors beyond their control. 

Another example is the chest thumping that some Americans 
engaged in after the fall of the Soviet Union. According to some, the 
demise of the USSR was caused by strong US policies, such as 
increased defense expenditures and the Strategic Defense Initiative, 
which caused Soviet leaders to realize they could no longer compete 
w.th the United States. The US news media played this story for sev- 
eral weeks, interviewing many people—some experts, some not—on 
why the Soviet Union collapsed. Most serious students understood 


'* It follows from the same reasoning that we may underestimate the consequences of our actions 
on nations that are not the iniended target of our influence. 
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that there were many reasons for the Soviet collapse, the most impor- 
tant of which were internal problems caused by the nature of the 
Soviet system. 

People and governments also tend to overestimate their own 
importance as the target of others’ actions. They are sensitive to the 
impact that others’ actions have on them, and they generally assume 
that people and governments intend to do what they do and intend it 
to have the effect that it has. They are much less aware of, and conse- 
quently tend to downgrade the importance of, other causes or results 
of the action. 

In analyzing the reasons why others act the way they do, it is 
common to ask, “What goals are the person or government pursuing?” 
But goals are generally inferred from the effects of behavior, and the 
effects that are best known and often seem most important are the 
effects upon ourselves. Thus actions that hurt us are commonly inter- 
preted as intentional expressions of hostility directed at ourselves. Of 
course, this will often be an accurate interpretation, but people some- 
times fail to recognize that actions that seem directed at them are actu- 
ally the unintended consequence of decisions made for other reasons. 


Iilusory Correlation 


At the start of this chapter, covariation was cited as one basis for 
inferring causality. It was noted that covariation may either be 
observed intuitively or measured statistically. This section examines 
the extent to which the intuitive perception of covariation deviates 
from the statistical measurement of covariation. 

Statistical measurement of covariation is known as correlation. 
Two events are correlated when the existence of one event implies the 
existence of the other. Variables are correlated when a change in one 
variable implies a similar degree of change in another. Correlation 
alone does not necessarily imply causation. For example, two events 
might co-occur because they have a common cause, rather than 
because one causes the other. But when two events or changes do co- 
occur, and the time sequence is such that one always follows the other, 
people ofte: infer that the first caused the second. Thus, inaccurate 
perception of correlation leads to inaccurate perception of cause and 
effect. 
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Judgments about correlation are fundamental to all intelligence 
analysis. For example, assumptions that worsening economic condi- 
tions lead to increased political support for an opposition party, that 
domestic problems may lead to foreign adventurism, that military 
government leads to unraveling of democratic institutions, or that 
negotiations are more successful when conducted from a position of 
strength are all based on intuitive judgments of correlation between 
these variables. In many cases these assumptions are correct, but they 
are seldom tested by systematic observation and statistical analysis. 

Much intelligence analysis is based on common-sense assump- 
tions about how people and governments normally behave. The prob- 
lem is that people possess a great facility for invoking contradictory 
“laws” of behavior to explain, predict, or justify different actions 
occurring under similar circumstances. “Haste makes waste” and “He 
who hesitates is lost” are examples of inconsistent explanations and 
admonitions. They make great sense when used alone and leave us 
looking foolish when presented together. “Appeasement invites 
aggression” and “agreement is based upon compromise” are similarly 
contradictory expressions. 

When confronted with such apparent contradictions, the natural 
defense is that “it all depends on. ...” Recognizing the need for such 
qualifying statements is one of the differences between subconscious 
information processing and systematic, self-conscious analysis. 
Knowledgeable analysis might be identified by the ability to fill in the 
qualification; careful analysis by the frequency with which one 
remembers to do so.'”’ 

Illusory correlation occurs when people perceive a relationship 
that does not in fact exist. In looking at a series of cases, it seems that 
people often focus on instances that support the existence of a rela- 
tionship but ignore those cases that fail to support it. Several experi- 
ments have demonstrated that people do not have an intuitive 
understanding of what information is really needed to assess the rela- 
tionship between two events or two variables. There appears to be 
nothing in people’s intuitive understanding that corresponds with the 
statistical concept of correlation. 


'’ This paragraph draws heavily from the ideas and phraseology of Baruch Fischhoff, “For Those 
Condemned to Study the Past: Reflections on Historical Judgment,” in R. A. Shweder and D. W. 
Fiske, eds., New Directions for Methodology of Behavioral Science: Fallible Judgment in Behav- 
ioral Research (San Francisco: Jossey-Bass, 1980). 
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Nurses were tested on their ability to learn through experience to 
judge the relationship, or correlation, between a symptom and the 
diagnosis of illness.'* The nurses were each shown 100 cards; every 
card ostensibly represented one patient. The cards had a row of four 
letters at the top representing various symptoms and another row of 
four letters at the bottom representing diagnoses. The nurses were 
instructed to focus on just one letter (A) representing one symptom 
and one letter (F) representing one diagnosis, and then to judge 
whether the symptom A was related to the diagnosis F. In other 
words, on the basis of experience with these 100 “patients,” does the 
presence of symptom A help to diagnose the presence of illness F? 
The experiment was run a number of times using different degrees of 
relationship between A and F. 

Put yourself briefly in the position of a test subject. You have 
gone through the cards and noticed that on about 25 of them, or a 
quarter of the cases, the symptom and the disease, A and F, are both 
present. Would you say there is a relationship? Why? Is it appropriate 
to make a judgment solely on the basis of the frequency of cases 
which support the hypothesis of a relationship between A and F? 
What else do you need to know? Would it be helpful to have the num- 
ber of cases in which the symptom (A) was present without the dis- 
ease (F)? Let us say this was also true on 25 cards, so that out of the 
100 cards, 50 had A and 25 of those cards with A also had F. In other 
words, the disease was present in half the cases in which the symp- 
tom was observed. Js this sufficient to establish a relationship, or is it 
also necessary to know the number of times the disease was present 
without the symptom? 

Actually, to determine the existence of such a relationship, one 
needs information to fill all four cells of a 2 x 2 contingency table. 
Figure 16 shows such a table for one test run of this experiment. The 
table shows the number of cases of patients having each of four possi- 
ble combinations of symptom and disease. 

Eighteen of 19 test subjects given the 100 cards representing this 
particular combination of A and F thought there was at least a weak 
relationship, and several thought there was a strong relationship, when 
in fact, there is no correlation at all. More than half the test subjects 
based their judgment solely on the frequency of cases in which both A 
and F were present. This is the upper left cell of the table. These sub- 
jects were trying to determine if there was a relationship between A 


'* Jan Smedslund, “The Concept of Correlation in Adults,” Scandinavian Journal of Psychology, 
Vol. 4 (1963), 165-73. 
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Figure 16 


Not F 


and F. When looking through the cards, 25 percent of the cases they 
looked at were consistent with the belief that symptom and diagnosis 
were perfectly correlated; this appears to be a lot of evidence to sup- 
port the hypothesized relationship. Another smaller group of test sub- 
jects used somewhat more sophisticated reasoning. They looked at the 
total number of A cases and then asked in how many of these cases F 
was also present. This is the left side of the table in Figure 16. A third 
group resisted the basic concept of making a statistical generaliza- 
tion. When asked to describe their reasoning, they said that sometimes 
a relationship was present while in other cases it was not. 

Of the 86 test subjects involved in several runnings of this exper- 
iment, not a single one showed any intuitive understanding of the con- 
cept of correlation. That is, no one understood that to make a proper 
judgment about the existence of a relationship, one must have infor- 
mation on all four cells of the table. Statistical correlation in its most 
elementary form is based on the ratio of the sums of the frequencies in 
the diagonal cells of a 2 x 2 table. In other words, a predominance of 
entries along either diagonal represents a strong statistical relation- 
ship between the two variables. 

Let us now consider a similar question of correlation on a topic of 
interest to intelligence analysts. What are the characteristics of strate- 
gic deception and how can analysts detect it? In studying deception, 
one of the important questions is: what are the correlates of deception? 
Historically, when analysts study instances of deception, what else do 
they see that goes along with it, that is somehow related to deception, 
and that might be interpret as an indicator of deception? Are there cer- 
tain practices relating to deception, or circumstances under which 
deception is most likely to occur, that permit one to say, that, because 
we have seen x or y or z, this most likely means a deception plan is 
under way”? This would be comparable to a doctor observing certain 
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Figure 17 (26p x 11p) 


symptoms and concluding that a given disease may be present. This is 
essentially a problem of correlation. If one could identify several corre- 
lates of deception, this would significantly aid efforts to detect it. 

The hypothesis has been advanced that deception is most likely 
when the stakes are exceptionally high.'” If this hypothesis is correct, 
analysts should be especially alert for deception in such instances. 
One can cite prominent examples to support the hypothesis, such as 
Pearl Harbor, the Normandy landings, and the German invasion of the 
Soviet Union. It seems as though the hypothesis has considerable sup- 
port, given that it is so easy to recall examples of high stakes situa- 
tions in which deception was employed. But consider what it would 
take to prove, empirically, that such a relationship actually exists. Fig- 
ure 17 sets up the problem as a 2 x 2 contingency table. 


Figure 17 


Barton Whaley researched 68 cases in which surprise or decep- 
tion was present in strategic military operations between 1914 and 
1968.'” Let us assume that some form of deception, as well as sur- 
prise, was present in all 68 cases and put this number in the upper left 
cell of the table. How many cases are there with high stakes when 
deception was not used? That is a lot harder to think about and to find 
out about; researchers seldom devote much effort to documenting neg- 
ative cases, when something did not occur. Fortunately, Whaley did 
make a rough estimate that both deception and surprise were absent in 
one-third to one-half of the cases of “grand strategy” during this 
period, which is the basis for putting the number 35 in the lower left 
cell of Figure 17. 


'* Robert Axelrod, “The Ra ional Timing of Surprise,” World Politics, XXX1 (January 1979), pp. 
228-246. 

'” Barton Whaley, Stratagem: Deception and Surprise in War, (Cambridge. MA: Massachusetts 
Institute of Technology, unpublished manuscript, 1969), p. 247. 
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How common is deception when the stakes are not high? This is 
the upper right cell of Figure 17. Entries for this cell and the lower 
right cell are difficult to estimate; they require defining a universe of 
cases that includes low-stakes situations. What is a low-stakes situa- 
tion in this context? High-stakes situations are definable, but there is 
an almost infinite number and variety of low-stakes situations. 
Because of this difficulty, it may not be feasible to use the full 2 x 2 
table to analyze the relationship between deception and high stakes. 

Perhaps it is necessary to be content with only the left side of the 
Figure 17 table. But then we cannot demonstrate empirically that one 
should be more alert to deception in high-stakes situations, because 
there is no basis for comparing high-stakes and low-stakes cases. If 
deception is even more common in tactical situations than it is in high 
stakes strategic situations, then analysts should not be more inclined 
to suspect deceptior when the stakes are high. 

It is not really clear whether there is a relationship between 
deception and high-stakes situations, because there are not enough 
data. Intuitively, your gut feeling may tell you there is, and this feeling 
may well be correct. But you may have this feeling mainly because 
you are inclined to focus only on those cases in the upper left cell that 
do suggest such a relationship. People tend to overlook cases where the 
relationship does not exist, inasmuch as these are much less salient. 

The lesson to be learned is not that analysts should do a statisti- 
cal analysis of every relationship. They usually will not have the data, 
time, or interest for that. But analysts should have a general under- 
standing of what it takes to know whether a relationship exists. This 
understanding is definitely not a part of people’s intuitive knowledge. 
It does not come naturally. It has to be learned. When dealing with 
such issues, analysts have to force themselves to think about all four 
cells of the table and the data that would be required to fill each cell. 

Even if analysts follow these admonitions, there are several fac- 
tors that distort judgment when one does not follow rigorous scien- 
tific procedures in making and recording observations. These are 
factors that influence a person’s ability to recall examples that fit into 
the four cells. For example, people remember occurrences more 
readily than non-occurrences. “History is, by and large, a record of 
what people did, not what they failed to do.”""' 

Thus, instances in which deception occurred are easier to recall 
than instances in which it did not. Analysts remember occurrences 
that support the relationship they are examining | etter than those that 


' E. H. Carr, What is History? (London: Macmillan, 1961), p. 126, cited by Fischhoff, op. cit. 
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do not. To the extent that perception is influenced by expectations, 
analysts may have missed or discounted the contrary instances. People 
also have a better memory for recent events, events in which they 
were personally involved, events that had important consequences, 
and so forth. These factors have a significant influence on perceptions 
of correlation when analysts make a gut judgment without consciously 
trying to think of all four cells of the table. 

Many erroneous theories are perpetuated because they seem plau- 
sible and because people record their experience in a way that sup- 
ports rather than refutes them. Ross describes this process as follows: 


..the intuitive observer selectively codes those data potentially 
relevant to the relationship between X and Y. Data points that fit 
his hypotheses and predictions are accepted as reliable, valid, 
representative, and free of error or “third-variable influences.” 
Such data points are seen as reflective of the “real” ... relationship 
between X and Y. By contrast, data points that deviate markedly 
from the intuitive ... expectations or theory are unlikely to be 
given great weight and tend to be dismissed as unreliable, errone- 
ous, unrepresentative, or the product of contaminating third-vari- 
able influences. Thus the intuitive scientist who believes that fat 
men are jolly, or more specifically that fatness causes jolliness, 
will see particular fat and jolly men as strong evidence for this 
theory; he will not entertain the hypothesis that an individual’s 
jollity is mere pretense or the product of a particularly happy 
home life rather than obesity. By contrast, fat and morose indi- 
viduals will be examined very carefully before gaining admis- 
sion to that scientist’s store of relevant data. He might, for 
instance, seek to determine whether the individual’s moroseness 
on the day in question is atypical, or the result of a nagging cold 
or a disappointing day, rather than the reflection of some stable 
attribute. It need hardly be emphasized that even a randomly gen- 
erated [set of data] can yield a relatively high conc!'a:ion if coded 
in the manner just outlined.'” 


' Ross, op. cit., pp. 208-209. 
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Chapter 12 


Biases in Estimating 
Probabilities 


In making rough probability judgments, people commonly 
depend upon one of several simplified rules of thumb that greatly ease 
the burden of decision. Using the “availability” rule, people judge the 
probability of an event by the ease with which they can imagine rele- 
vant instances of similar events or the number of such events that they 
can easily remember. With the “anchoring” strategy, people pick 
some natural starting point for a first approximation and then adjust 
this figure based on the results of additional information or analysis. 
Typically, they do not adjust the initial judgment enough. 

Expressions of probability, such as possible and probable, are a 
common source of ambiguity that make it easier for a reader to inter- 
pret a report as consistent with the reader's own preconceptions. The 
probability of a scenario is often miscalculated. Data on “prior prob- 
abilities” are commonly ignored unless they illuminate causal rela- 
tionships. 


ee eeegzaeeeeEe EE E * 


Availability Rule 


One simplified rule of thumb commonly used in making proba- 
bil’*y estimates is known as the availability rule. In this context, 
“availability” refers to imaginability or retrievability from memory. 
Psychologists have shown that two cues people use unconsciously in 
judging the probability of an event are the ease with which they can 
imagine relevant instances of the event and the number or frequency 
of such events that they can easily remember.'*’ People are using the 
availability rule of thumb whenever they estimate frequency or proba- 
bility on the basis of how easily they can recall or imagine instances 
of whatever it is they are trying to estimate. 


‘Amos Tversky and Daniel Kahneman, “Availability: A Heuristic for Judging Frequency and 
Probability.” Cognitive Psychology, 5 (1973), pp. 207-232. 
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Normally this works quite well. If one thing actually occurs 
more freque.tly than another and is therefore more probable, we 
probably can recall more instances of it. Events that are likely to 
occur usually are easier to imagine than unlikely events. People are 
constantly making inferences based on these assumptions. For exam- 
ple, we estimate our chances for promotion by recalling instances of 
promotion among our colleagues in similar positions and with similar 
experience. We estimate the probability that a politician will lose an 
election by imagining ways in which he may lose popular support. 

Although this often works well, people are frequently led astray 
when the ease with which things come to mind is influenced by fac- 
tors unrelated to their probability. The ability to recall instances of an 
event is influenced by how recently the event occurred, whether we 
were personally involved, whether there were vivid and memorable 
details associated with the event, and how important it seemed at the 
time. These and other factors that influence judgment are unrelated to 
the true probability of an event. 

Consider two people who are smokers. One had a father who 
died of lung cancer, whereas the other Jes not know anyone who 
ever had lung cancer. The one whose father died of lung cancer will 
normally perceive a greater probability of adverse health conse- 
quences associated with smoking, even though one more case of lung 
cancer is statistically insignificant when weighing such risk. How 
about two CIA officers, one of whom knew Aldrich Ames and the 
other who did not personally know anyone who had ever turned out to 
be a traitor? Which one is likely to perceive the greatest risk of insider 
betrayal? 

It was difficult to imagine the breakup of the Soviet Union 
because such an event was so foreign to our experience of the previ- 
ous 50 years. How difficult is it now to imagine a return to a Commu- 
nist regime in Russia? Not so difficult, in part because we still have 
vivid memories of the old Soviet Union. But is that a sound basis for 
estimating the likelihood of its happening? When analysts make 
quick, gut judgments without really analyzing the situation, they are 
likely to be influenced by the availability bias. The more a prospec- 
live scenario accords with one’s experience, the easier it is to imagine 
and the more likely it seems. 

Intelligence analysts may be less influenced than others by the 
availability bias. Analysts are evaluating all available information, not 
making quick and easy inferences. On the other hand, policymakers 
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and journalists who lack the time or access to evidence to go into 

details must necessarily take shortcuts. The obvious shortcut is to use 

the availability rule of thumb for making inferences about probability. 
Many events of concern to intelligence analysts 


... are perceived 2 so unique that past history does not seem rel- 
evant to the eva! ation of their likelihood. In thinking of such 
events we often construct scenarios, i.e., stories that lead from 
the present situation to the target event. The plausibility of the 
scenarios that come to mind, or the difficulty of producing them, 
serve as clues to the likelihood of the event. If no reasonable sce- 
nario comes to mind, the event is deemed impossible or highly 
unlikely. If several scenarios come easily to mind, or if one sce- 
nario is particularly compelling, the event in question appears 


probable.'“ 


US policymakers in the early years of our involvement in Viet- 
nam had to imagine scenarios for what might happen if they did or did 
not commit US troops to the defense of South Vietnam. In judging the 
probability of alternative outcomes, our senior leaders were strongly 
influenced by the ready availability of two seemingly comparable sce- 
narios—the failure of appeasement prior to World War II and the suc- 
cessful intervention in Korea. 

Many extraneous factors influence the imaginability of scenar- 
ios for future events, just as they influence the retrievability of events 
from memory. Curiously, one of these is the act of analysis itself. The 
act of constructing a detailed scenario for a possible future event 
makes that event more readily imaginable and, therefore, increases its 
perceived probability. This is the experience of CIA analysts who 
have used various tradecraft tools that require, or are especially suited 
to, the analysis of unlikely but nonetheless possible and important 
hypotheses. (Such techniques were discussed in Chapter 6, “Keeping 
an Open Mind” and Chapter 8, “Analysis of Competing Hypothe- 
ses.”) The analysis usually results in the “unlikely” scenario being 
taken a little more seriously. This phenomenon has also been demon- 
strated in psychological experiments. '* 


™ Tbid., p. 229. 

'* John S. Carroll, “The Effect of Imagining an Event on Expectations for the Event: An Interpre- 
tation in Terms of the Availability Heuristic, Journal of Experimental Social Psychology, \4 
(1978), pp. 88-96. 
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In sum, the availability rule of thumb is often used to make judg- 
ments about likelihood or frequency. People would be hard put to do 
otherwise, inasmuch as it is such a timesaver in the many instances 
when more detailed analysis is not warranted or not feasible. Intelli- 
gence analysts, however, need to be aware when they are taking short- 
cuts. They must know the strengths and weaknesses of these 
procedures, and be able to identify when they are most likely to be led 
astray. For intelligence analysts, recognition that they are employing 
the availability rule should raise a caution flag. Serious analysis of 
probability requires identification and assessment of the strength and 
interaction of the many variables that will determine the outcome of a 
situation. 


Anchoring 


Another strategy people seem to use intuitively and uncon- 
sciously to simplify the task of making judgments is called anchor- 
ing. Some natural starting point, perhaps from a previous analysis of 
the same subject or from some partial calculation, is used as a first 
approximation to the desired judgment. This starting point is then 
adjusted, based on the results of additional information or analysis. 
Typically, however, the starting point serves as an anchor or drag that 
reduces the amoi ..i of adjustment, so the final estimat: remains closer 
to the starting point than it ought to be. 

Anchoring can be demonstrated very simply in a classroom exer- 
cise by asking a group of students to estimate one or more known 
quantities, such as the percentage of member countries in the United 
Nations that are located in Africa. Give half the students a low-per- 
centage number and half a high-percentage number. Ask them to start 
with this number as an estimated answer, then, as they think about the 
problem, to adjust this number until they get as close as possible to 
what they believe is the correct answer. When this was done in one 
experiment that used this question, those starting with an anchor of 10 
percent produced adjusted estimates that averaged 25 percent. Those 
who started with an anchor of 65 percent produced adjusted estimates 
that averaged 45 percent.'” 


‘* Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” 
Science, Vol. 185, Sept. 27, 1974, pp. 1124-1131. 
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Because of insufficient adjustment, those who started out with an 
estimate that was too high ended with significantly higher estimates 
than those who began with an estimate that was too low. Even the 
totally arbitrary starting points acted as anchors, causing drag or iner- 
tia that inhibited full adjustment of estimates. 

Whenever analysts move into a new analytical area and take over 
responsibility for updating a series of judgments or estimates made by 
their predecessors, the previous judgments may have such an anchor- 
ing effect. Even when analysts make their own initial judgment, and 
then attempt to revise this judgment on the basis of new information 
or further analysis, there is much evidence to suggest that they usu- 
ally do not change the judgment enough. 

Anchoring provides a partial explanation of experiments show- 
ing that analysts tend to be overly sure of themselves in setting confi- 
dence ranges. A military analyst who estimates future missile or tank 
production is often unable to give a specific figure as a point estimate. 
The analyst may, therefore, set a range from high to low, and estimate 
that there is, say, a 75-percent chance that the actual production figure 
will fall within this range. If a number of such estimates are made that 
reflect an appropriate degree of confidence, the true figure should fall 
within the estimated range 75 percent of the time anc outside this 
range 25 percent of the time. In experimental situations, however, 
most participants are overconfident. The true figure falls outside the 
estimated range a much larger percentage of the time.'”’ 

If the estimated range is based on relatively hard information 
concerning the upper and lower limits, the estimate is likely to be 
accurate. If, however, the range is determined by starting with a sin- 
gle best estimate that is simply adjusted up and down to arrive at esti- 
mated maximum and minimum values, then anchoring comes into 
play, and the adjustment is likely to be insufficient. 

Reasons for the anchoring phenomenon are not well understood. 
The initial estimate serves as a hook on which people hang their first 
impressions or the results of earlier calculations. In recalculating, they 
take this as a starting point rather than starting over from scratch, but 
why this should limit the range of subsequent reasoning 1s not clear. 


Experiments using a 98-percent confidence range found that the true value fell outsice the esti- 
mated range 40 to 50 percent of the time. Amos Tversky and Damel Kahneman, “Anchoring and 
Calibration in the Assessment of Uncertain Quantities.” (Oregon Research Institute Research Bul- 
letin, 1972, Nov. 12, No. 5), and M. Alpert and H. Raiffa, “A Progress Report on The Training of 
Probability Assessors,” Unpublished manuscript. Harvard University, 1968. 


There is some evidence that awareness of the anchoring prob- 
lem is not an adequate antidote.'* This is a common finding in experi- 
ments dealing with cognitive biases. The biases persist even after test 
subjects are informed of them and instructed to try to avoia them or 
compensate for them. 

One technique for avoiding the anchoring bias, to weigh anchor 
so to speak, may be to ignore one’s own or others’ earlier judgments 
and rethink a problem from scratch. In other words, consciously avoid 
any prior judgment as a starting point. There is no experimental evi- 
dence to show that this is possible or that it will work, but it seems 
worth trying. Alternatively, it is sometimes possible to avoid human 
error by employing formal statistical procedures. Bayesian statistical 
analysis, for example, can be used to revise prior judgments on the 
basis of new information in a way that avoids anchoring bias.'” 


Expression of Uncertainty 


Probabilities may be expressed in two ways. Statistical probabil- 
ities are based on empirical evidence concerning relative frequencies. 
Most intelligence judgments deal with one-of-a-kind situations ior 
which it is impossible to assign a statistical probability. Another 
approach commonly used in intelligence analysis is to make a “sub- 
jective probability” or “personal probability” judgment. Such a judg- 
ment is an expression of the analyst’s personal belief that a certain 
explanation or estimate is correct. It is comparable to a judgment that 
a horse has a three-to-one chance of winning a race. 

Verbal expressions of uncertainty—such as “possible,” “proba- 
ble,” “unlikely,” “may,” and “could”—are a form of subjective proba- 
bility judgment, but they have long been recognized as sources of 
ambiguity and misunderstanding. To say that something could happen 


'* Alpert and Raiffa, ibid. 

‘’ Nicholas Schweitzer, “Bayesian Analysis: Estimating the Probability of Middle East Conflict,” 
in Richards J. Heuer, Jr., ed., Quantitative Approaches to Political Intelligence: The CIA Experi- 
ence (Bo ilder, CO: Westview Press, 1979). Jack Zlotnick, “Bayes Theorem for Intelligence Anal- 
ysis.” Studies in Intelligence, Yol. 16, No. 2 (Spring 1972). Charles E. Fisk, “The Sino-Soviet 
Border Dispute: A Comparison of che Conventional and Bayesian Methods for Intelligence Warn- 
ing, Studies in Intellisence, vol. 16, no. 2 (Spring 1972), originally classified Secret, now declassi- 
fied. Both the Zlotnick and Fisk articles were republished in H. Bradford Westerfield, Inside CIA's 
Private World: Declassijed Articles from the Agency's Internal Journal, 1955-1992, (New Haven: 
Yale University Press, 1995). 
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or is possible may refer to anything from a |-percent to a 99-percent 
probability. To express themselves clearly, analysts must learn to rou- 
tinely communicate uncertainty using the language of numerical prob- 
ability or o ‘ds ratios. 

As explained in Chapter 2 on “Perception,” people tend to see 
what they expect to see, and new information is typically assimilated 
to existing beliefs. This is especially true when dealing with verbal 
expressions of uncertainty. By themselves, these expressions have no 
clear meaning. They are empty shells. The reader or listener fills them 
with meaning through the context in which they are used and what is 
already in the reader’s or listener’s mind about that context. 

When intelligence corclusions are couched in ambiguous terms, 
a reader’s interpretation of the conclusions will be biased in favor of 
consistency with what the reader already believes. This may be one 
reason why many intelligence consumers say they do not learn much 
from intelligence reports. '“ 

It is easy to demonstrate this phenomenon in training courses for 
analysts. Give students a short intelligence report, have them under- 
line all expressions of uncertainty, then have them express their under- 
standing of the report by writing above each expression of uncertainty 
the numerical probability they believe was intended by the writer of 
the report. This is an excellent learning experience, as the differences 
among students in how they understand the report are typically so 
great as to be quite memorable. 

In one experiment, an intelligence analyst was asked to substi- 
tute numerical probability estimates for the verbal qualifiers in one of 
his own earlier articles. The first statement was: “The cease-fire is 
holding but could be broken within a week.” The analyst said he 
meant there was about a 30-percent chance the cease-fire would be 
broken within a week. Another analyst who had helped this analyst 
prepare the article said she thought there was about an 80-percent 
chance that the cease-fire would be broken. Yet, when working 
together on the report, both analysts had believed they were in agree- 
ment about what could happen.'' Obviously, the analysts had not even 
communicated effectively with each other, let alone with the readers 
of their report. 


“ For another interpretation of this phenomenon, see Chapter | 3, “Hindsight Biases in Evaluation 
of Intelligence Reporting.” 

* Scott Barclay et al, Handbook for Decision Analysis. (McLean, VA: Decisions and Designs, Inc. 
1977), p. 66. 
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Sherman Kent, the first director of CIA’s Office of National Esti- 
mates, was one of the first to recognize problems of communication 
caused by imprecise statements of uncertainty. Unfortunately, several 
decades after Kent was first jolted by how policymakers interpreted 
the term “serious possibility” in a national estimate, this miscommuni- 
cation between analysts and policymakers, and between analysts, is 
still a common occurrence. '” 

I personally recall an ongoing debate with a colleague over the 
bona fides of a very important source. I argued he was probably bona 
fide. My colleague contended that the source was probably under hos- 
tile control. After several months of periodic disagreement, I finally 
asked my colleague to put a number on it. He said there was at least a 
51-percent chance of the source being under hostile control. | said 
there was at least a 51-percent chance of his being bona fide. Obvi- 
ously, we agreed that there was a great deal of uncertainty. That 
stopped our disagreement. The problem was not a major difference of 
opinion, but the ambiguity of the term probable. 

The table in Figure 18 shows the results of an experiment with 
23 NATO military officers accustomed to reading intelligence reports. 
They were given a number of sentences such as: “It is highly unlikely 
that ....” All the sentences were the same except that the verbal 
expressions of probability changed. The officers were asked what per- 
centage probability they would attribute to each statement if they read 
it in an intelligence report. Each dot in the table represents one 
Officer’s probability assignment.'** While there was broad consensus 
about the meaning of “better than even,” there was a wide disparity in 
interpretation of other probability expressions. The shaded areas in the 
table show the ranges proposed by Kent.'“ 

The main point is that an intelligence report may have no impact 
on the reader if it is couched in such ambiguous language that the 
reader can easily interpret it as consistent with his or her own precon- 
ceptions. This ambiguity can be especially troubling when dealing 
with low-probability, high-impact dangers against which policymakers 
may wish to make contingency plans. 


‘© Sherman Kent, “Words of Estimated Probability,” in Donald P. Steury, ed., Sherman Kent and 
the Board of National Estimates: Collected Essays (CIA, Center for the Study of Intelligence, 
1994). 

*’ Scott Barclay et al, p. 76-68. 

'“ Probability ranges attributed to Kent in this table are slightly different from those in Sherman 
Kent, “Words of Estimated Probability,” in Donald P. Steury, ed., Sherman Kent and the Board of 
National Estimates: Collected Essays (CIA, Center for the Study of Intelligence, 1994). 
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Figure 18: Measuring Perceptions of Uncertainty 
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Consider, for example, a report that there is little chance of a ter- 
rorist attack against the American Embassy in Cairo at this time. If the 
Ambassador’s preconcepiion is that there is no more than a one-in-a- 
hundred chance, he may eiect to not do very much. If the Ambassa- 
dor’s preconception is that there may be as much as a one-in-four 
chance of an attack. lhe may decide to do quite a bit. The term “little 
chance” is consistent with either of those interpretations, and there is 
no way to know what the report writer meant. 

Another potential ambiguity is the phrase “‘at this time.” Shorten- 
ing the time frame for prediction lowers the probability, but may not 
decrease the need for preventive measures or contingency planning. 
An event for which the timing is unpredictable may “at this time” 
have only a 5 percent probability of occurring during the coming 
month, but a 60-percent probability if the time frame is extended to 
one year (5 ;ercent per month for 12 months). 

How can analysts express uncertainty without being unclear 
about how certain they are? Putting a numerical qualifier in parenthe- 
ses after the phrase expressing degree of uncertainty is an appropriate 
means of avoiding misinterpretation. This may be an odds ratio (less 
than a one-in-four chance) or a percentage range (5 to 20 percent) or 
(less than 20 percent). Odds ratios are often preferable, as most people 
have a better intuitive understanding of odds than of percentages. 


Assessing Probability of a Scenario 


Intelligence analysts sometimes present judgments in the form of 
a scenario—a series of events leading to an anticipated outcome. 
There is evidence that judgments concerning the probability of a sce- 
nario are influenced by amount and nature of cetail in the scenario in 
a way that is unrelated to actual likelihood of ihe scenario. 

A scenario cunsists of several events iinked together in a narra- 
tive description. To calculate mathematically the probability of a sce- 
nario, the proper procedure is to multiply the probabilities of each 
individual event. Thus, for a scenario with three events, each of which 
will probably (70 percent certainty) occur, the probability of the sce- 
nario is .70 x .70 x .70 or slightly over 34 percent. Adding a fourth 
probable (70 percent) event to the scenario would reduce its probabil- 
ity to 24 percent. 


Most people do not have a good intuitive grasp of probabilistic 
reasoning. One approach to simplifying such problems is to assume 
(or think as though) one or more probable events have already 
occurred. This eliminates some of the uncertainty from the judgment. 
Another way to simplify the problem is to base judgment on a rough 
average of the probabilities of each event. In the above example, the 
averaging procedure gives an estimated probability of 70 percent for 
the entire scenario. Thus, the scenario appears far more likely than is 
in fact the case. 

When the averaging strategy is employed, highly probable 
events in the scenario tend to offset less probable events. This vio- 
lates the principle that a chain cannot be stronger than its weakest 
link. Mathematically, the least probable event in a scenario sets the 
upper limit on the probability of the scenario as a whole. If the aver- 
aging strategy is employed, additional details may be added to the 
scenario that are so plausible they increase the perceived probability, 
of the scenario, while, mathematically, additional events must neces- 
sarily reduce its probability.'" 


Base-Rate Fallacy 

In assessing a situation, an analyst sometimes has two kinds of 
evidence available—specific evidence about the individual case at 
hand, and numerical data that summarize information about many 
similar cases. This type of numerical information is called a base rate 
or prior probability. The base-rate fallacy is that the numerical data 
are commonly ignored unless they illuminate a causal relationship. 
This is illustrated by the following experiment. '* 

During the Vietnam War, a fighter plane made a non-fatal straf- 
ing attack on a US aerial reconnaissance mission at twilight. Both 
Cambodian and Vietnamese jets operate in the area. You know the fol- 
lowing facts: 

(a) Specific case information: The US pilot identified the fighter 
as Cambodian. The pilot’s aircraft recognition capabilities were tested 
under appropriate visibility .nd flight conditions. When presented 
with a sample of fighters (half with Vietnamese markings and half 
with Cambodian) the pilot made correct identifications 80 percent of 
the time and erred 20 percent of the time. 


“* Paul Slovic, Baruch Fischhoff, and Sarah Lichtenstein, “Cognitive Processes and Societal Risk 
Taking,” in J. S. Carroll and J.W. Payne, eds., Cognition and Social Behavior Potomac, MD: 
Lawrence Erlbaum Associates, 1976), pp. 177-78. 

“ This is a modified version, developed by Frank J. Stech, of the blue and green taxicab question 
used by Kahneman and Tversky, “On Prediction and Judgment,’ Oregon Research Institute 
Research Bulletin, 12, 14, 1972. 
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(b) Base rate data: 85 percent of the jet fighters in that area are 
Vietnamese, 15 percent are Cambodian. 

Question: What is the probability that the fighter was Cambo- 
dian rather than Vietnamese? 

A common procedure in answering this question is to reason as 
follows: We know the pilot identified the aircraft as Cambodian. We 
also know the pilot's identifications are correct 80 percent of the time; 
therefore, there is an 80 percent probability the fighter was Cambo- 
dian. This reasoning appears plausible but is incorrect. It ignores the 
base rate—that 85 percent of the fighters in that area are Vietnamese. 
The base rate, or prior probability, is what you can say about any hos- 
tile fighter in that area before you learn anything about the specific 
sighting. 

It is actually more likely that the plane was Vietnamese than 
Cambodian despite the pilot's “probably correct” identification. Read- 

ers who are unfamiliar with probabilistic reasoning and do not grasp 
“this point should imagine 100 cases in which the pilot ha a simifar’ ” 
encounter. Based on paragraph (a), we know that 80 percent or 68 of 
the 85 Vietnamese aircraft will be correctly identified as Vietnamese, 
while 20 percent or 17 wiil be incorrectly identified as Cambodian. 
Based on paragraph (b), we know that 85 of these encounters will be 
with Vietnamese aircraft, 15 with Cambodian. 

Similarly, 80 percent or 12 of the 15 Cambodian aircraft will be 
correctly identified as Cambodian, while 20 percent or three will be 
incorrectly identified as Vietnamese. This makes a total of 71 Viet- 
namese and 29 Cambodian sightings, of which only 12 of the 29 
Cambodian sightings are correct; the other 17 are incorrect sightings 
of Vietnamese aircraft. Therefore, when the pilot claims the attack 
was by a Cambodian fighter, the probability that the craft was actu- 
ally Cambodian is only 12/29ths or 41 percent, despite the fact that 
the pilot's identifications are correct 80 percent of the time. 

This may seem like a mathematical trick, but it is not. The dif- 
ference stems from the strong prior probability of the pilot observing 
a Vietnamese aircraft. The difficulty in understanding this arises 
because untrained intuitive judgment does not incorporate some of the 
basic statistical principles of probabilistic reasoning. Most people do 
not incorporate the prior probability into their reasoning because it 
does not seem relevant. It does not seem relevant because there is no 
causal relationship between the background information on the per- 
centages of jet fighters in the area and the pilot’s observation.'”’ The 


* Maya Bar-Hillel, “The Base-Rate Fallacy in Probability Judgments,” Acta Psychologica, 1°80. 
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fact that 85 percent of the fighters in the area were Vietnamese and 15 
percent Cambodian did not cause the attack to be made by a Cambo- 
dian rather than a Vietnamese. 

To appreciate the different impact made by causally relevant 
background information, consider this alternative formulation of the 
same problem. In paragraph (b) of the problem, substitute the follow- 
ing: 

(b) Although the fighter forces of the two countries are rougiily 
equal in number in this area, 85 percent of all harassment incidents 
involve Vietnamese fighters, while 15 percent involve Cambodian 
fighters. 

The problem remains mathematically and structurally the same. 
Experiments with many test subjects, however, show it is quite differ- 
ent psychologically because it readily elicits a causal explanation 
relating the prior probabilities to the pilot's observation. If the Viet- 
namese have a propensity to harass and the Cambodians do not, the 
‘priof probability that Vietnamese ‘harassment ts more*tikely than Cam- 
bodian is no longer ignored. Linking the prior probability to a cause 
and effect relationship immediately raises the possibility that the 
pilot's observation was in error. 

With this revised form iation of the problem, most people are 
likely to reason as follows: We know from past experience in cases 
such as this that the harassment is usually done by Vietnamese air- 
craft. Yet, we have a fairly reliable report from our pilot that it was a 
Cambodian fighter. These two conflicting pieces of evidence cance! 
each other out. Therefore, we do not know—it is roughly 50-50 
whether it was Cambodian or Vietnamese. In employing this reason- 
ing, we use the prior probability information, integrate it with the 
case-specific information, and arrive at a conclusion that is about as 
close to the optimal answer (still 41 percent) as one is going to get 
without doing a mathematical calculation. 

There are, of course, few problems in which base rates are given 
as explicitly as in the Vietnamese/Cambodian aircraft example. When 
base rates are not well known but must be inferred or researched, they 
are even less likely tu be used.'* 

The so-called planning fallacy, to which I personally plead 
guilty, is an example of a problem in which base rates are not given in 
numerical terms but must be abstracted from experience. 'n planning a 


“ Many examples from everyday life are cited in Robyn M. Dawes, Rational Choice in an Uncer- 
tain World (Harcourt Brace Jovanovich College Publishers, 1988), Chapter 5. 
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research project, | may estimate being able to complete it in four 
weeks. This estimate is based on relevant case-specific evidence: 
desired length of report, availability of source materials, difficulty of 
the subject matter, allowance for both predictable and unforesee: >le 
interruptions, and so on. I also possess a body of experience with sim- 
ilar estimates I have made in the past. Like many others, I almost 
never complete a research project within the initially estimated time 
frame! But I am seduced by the immediacy and persuasiveness of the 
case-specific evidence. All the causally relevant evidence about the 
project indicates I should be able to complete the work in the time 
allotted tor it. Even though I know from experience that this never 
happens, I do not learn from this experience. I continue to ignore the 
non-causal, probabilistic evidence based on many similar projects in 
the past, and to estimate completion dates that I hardly ever meet. 
(Preparation of this book took twice as long as I had anticipated. 
These biases are, indeed, difficult to avoid!) 
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Chapter 13 


Hindsight Biases in Evaluation 
of Intelligence Reporting 


Evaluations of intelligence analysis—analysts'’ own evaluations 
«their judgments as well as others’ evaluations of intelligence prod- 
ucts—are distorted by systematic biases. As a result, analysts overes- 
timate the quality of their analytical performance, and others 
underestimate the value and quality of their efforts. These biases are 
not simply the product of self-interest and lack of objectivity. They 
stem from the nature of human mental processes and are diffi cult and 
Perhhps impossible to dvercome.™ * — 
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Hindsight biases influence the evaluation of intelligence report- 
ing in three ways: 


¢ Analysts normally overestimate the accuracy of their past judg- 
ments. 


¢ Intelligence consumers normally underestimate how much they 
learned from intelligence reports. 


¢ Overseers of intelligence production who conduct postmortem 
analyses of an intelligence failure normally judge that events 
were more readily foreseeable than was in fact the case. 


None of the biases is surprising. Analysts have observed these 
tendencies in others, although probably not in themselves. What may 
be unexpected is thai these biases are not only the product of self- 
interest and lack of objectivity. They are examples of a broader phe- 
nomenon that is built into human mental processes and that cannot be 
overcome by the simple admonition to be more objective. 


This chapte =. ¢: — published as an unclassified article in Studies in Intelligence, Vol. 22, No. 
2 (Summer 1S 5). o: ter the tithe “Cognitive Biases: Problems in Hindsight Analysis.” It was later 
published in [) &s¢cciord Westerfield, editor, Inside CIA's Private World: Declassified Articles 
from the Agenc’ » internal Journal, 1955-1992 (New Haven: Yale University Press, 1995.) 
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Psychologists who conducted the experiments described below 
tried to teach test subjects to overcome these biases. Experimental 
subjects with no vested interest in the results were briefed on the 
biases and encouraged to avoid them or compensate for them, but 
could not do so. Like optical illusions, cognitive biases remain com- 
pelling even after we become aware of them. 

The analyst, consumer, and overseer evaluating analytical perfor- 
mance all have one thing in common. They are exercising hindsight. 
They take their current state of knowledge and compare it with what 
they or others did or could or should have known before the current 
knowledge was received. This is in sharp contrast with intelligence 
estimation, which is an exercise in foresight, and it is the difference 
between these two modes of thought—hindsight and foresight—that 
seems to be a source of bias. 

The amount of good information that is available obviously is 

2 . renter in hjqdsigbt shan.in foresight Thera are several possihle.expla-. . . .. 
nations of how this affects mental processes. One is that the additional 
information available for hindsight changes perceptions of a situation 
so naturally and so immediately that people are largely unaware of the 
change. When new information is received, it is immediately and 
unconsciously assimilated into our pre-existing knowledge. If this new 
information adds significantly to our knowledge—that is, if it tells the 
outcome of a situation or the answer to a question about which we 
were previously uncertain—our mental images are restructured to take 
the new information into account. With the benefit of hindsight, for 
example, factors previously considered relevant may become irrele- 
vant, and factors previously thought to have little relevance may be 
seen as determinative. 

After a view has been restructured to assimilate the new infor- 
mation, there is virtually no way to accurately reconstruct the pre- 
existing mental set. Once the bell has rung, it cannot be unrung. A 
person may remember his or her previous judgments if not much time 
has elapsed and the judgments were precisely articulated, but appar- 
ently people cannot accurately reconstruct their previous thinking. 
The effort to reconstruct what we previously thought about a given sit- 
uation, or what we would have thought about it, is inevitably influ- 
enced by our current thought patterns. Knowing the outcome of a 
situation makes it harder to imagine other outcomes that might have 
been considered. Unfortunately, simply understanding that the mind 
works in this fashion does little to help overcome the limitation. 
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The overall message to be learned from an understanding of 
these biases, as shown in the experiments described below, is that an 
analyst's intelligence judgments are not as good as analysts think they 
are, or as bad as others seem to believe. Because the biases generally 
cannot be overcome, they would appear to be facts of life that analysts 
need to take into account in evaluating their own performance and in 
determining what evaluations to expect from others. This suggests the 
need for a more systematic effort to: 


¢ Define what should be expected from intelligence analysts. 


¢ Develop an institutionalized procedure for comparing intelli- 
gence judgments and estimates with actual outcomes. 


¢ Measure how well analysts live up to the defined expectations. 

.- ’ . --@- -o o-* » * . . 7? % see * 
The discussion now turns to the experimental evidence demon- 

strating these biases from the perspective of the analyst, consumer, 


and overseer of intelligence. 


The Analyst’s Perspective 


Analysts interested in improving their own performance need to 
evaluate their past estimates in the light of subsequent developments. 
To do this, analysts must either remember (or be able to refer to) their 
past estimates or must reconstruct their past estimates on the basis of 
what they remember having known about the situation at the time the 
estimates were made. The effectiveness of the evaiuation process, and 
of the learning process to which it gives impetus, depends in part upon 
the accuracy of these remembered or reconstructed estimates. 

Experimental evidence suggests a systematic tendency toward 
faulty memory of past estimates.'” That is, when events occur, people 
tend to overestimate the extent to which they had previously expected 
them to occur. And conversely, when events do not occur, people tend 
to underestimate the probability they had previously assigned to their 
occurrence. In short, events generally seem less surprising than they 


' This section is based on research reported by Baruch Fischoff and Ruth Beyth in “I Knew It 
Would Happen: Remembered Probabilities of Once-Future Things,” Organizational Behavior and 
Human Performance, |13 (1975), pp. 1-16. 


should on the basis of past estimates. This experimental evidence 
accords with analysts’ intuitive experience. Analysts rarely appear— 
or allow themselves to appear—very surprised by the course of events 
they are following. 

In experiments to test the bias in memory of past estimates, 119 
subjects were asked to estimate the probability that a number of 
events would or would not occur during President Nixon's trips to 
Peking and Moscow in 1972. Fifteen possible outcomes were identi- 
fied for each trip, and each subject assigned a probability to each of 
these outcomes. The outcomes were selected to cover the range of 
possible developments and to elicit a wide range of probability values. 

At varying time periods after the trips, the same subjects were 
asked to remember or reconstruct their own predictions as accurately 
as possible. (No mention was made of the memory task at the time of 
the original prediction.) Then the subjects were asked to indicate 
whether they thought each event had or had ‘not ‘occurred during these 
trips. 

When three to six months were allowed to elapse between the 
subjects’ estimates and their recollection of these estimates, 84 per- 
cent of the subjects exhibited the bias when dealing with events they 
believed actually did happen. That is, the probabilities they remem- 
bered having estimated were higher than their actual estimates of 
events they believed actually did occur. Similarly, for events they 
believed did not occur, the probabilities they remembered having esti- 
mated were lower than their actual estimates, although here the bias 
was not as great. For both kinds of events, the bias was more pro- 
nounced after three to six months had elapsed than when subjects 
were asked to recall estimates they had given only two weeks earlier. 

In summary, knowledge of the outcomes somehow affected most 
test subjects’ memory of their previous estimates of these outcomes, 
and the more time that was allowed for memories to fade, the greater 
the effect of the bias. The developments during the President’s trips 
were perceived as less surprising than they would have been if actual 
estimates were compared with actual outcomes. For the 84 percent of 
subjects who showed the anticipated bias, their retrospective evalua- 
tion of their estimative performance was clearly more favorable than 
warranted by the facts. 
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The Consumer’s Perspective 


When consumers of intelligence reports evaluate the quality of 
the intelligence product, they ask themselves the question: “How 
much did | learn from these reports that I did not already know?” In 
answering this question, there is a consistent tendency for most people 
to underestimate the contribution made by new information. This “I 
knew it all along” bias causes consumers to undervalue the intelli- 
gence product. '*' 

That people do in fact commonly react to new information in 
this manner was tested in a series of experiments involving some 320 
people, each of whom answered the same set of 75 factual questions 
taken from almanacs and encyclopedias. As a measure of their confi- 
dence in their answers, the subjects assigned to each question a num- 
ber ranging from 50 percent to 100 percent, indicating their estimate 
"of thie probability that they had chosen the correct answer. 

As a second step in the experiment, subjects were divided into 
three groups. The first group was given 25 of the previously asked 
questions and instructed to respond t. ‘hem exactly as they had previ- 
ously. This simply tested the subjects’ ability to remember their previ- 
ous answers. The second group was given the same set of 25 
questions but with the correct answers circled “for your [the subjects’ ] 
general information.” They, too, were asked to respond by reproduc- 
ing their previous answers. This tested the extent to which learning 
the correct answers distorted the subjects’ memories of their own pre- 
vious answers, thus measuring the same bias in recollection of previ- 
ous estimates that was discussed above from the analyst's perspective. 

The third group was given a different set of 25 questions they 
had not previously seen, but which were of similar difficulty so that 
results would be comparable with the other two groups. The correct 
answers were marked on the questionnaire, and the subjects were 
asked to respond to the questions as they would have responded had 
they not been told the answer. This tested their ability to recall accu- 
rately how much they had known before they learned the correct 
answer. The situation is comparable to that of intelligence consumers 


“' Experiments described in this section are reported in Baruch Fischhoff, The Perceived Informa- 
tiveness of Factual Information, Technical Report DDI- | (Eugene, OR: Oregon Research Institute, 
1976). 
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who are asked to evaluate how much they learned from a report, and 
who can do this only by trying to recollect the extent of their knowl- 
edge before they read the report. 

The most significant results came from this third group of sub- 
jects. The group clearly overestimated what they had known originally 
and underestimated how much they learned from having been told the 
answer. For 19 of 25 items in one running of the experiment and 20 of 
25 items in another running, this group assigned higher probabilities 
to the correct alternatives than it is reasonable to expect they would 
have assigned had they not already known the correct answers. 

In summary, the experiment confirmed the results of the previous 
experiment showing that people exposed to an answer tend to remem- 
ber having known more than they actually did. It also demonstrates 
that people have an even greater tendency to exaggerate the likeli- 
— dheod dhat they would -have known the ecorreet answer-if: they had net: 
been informed of it. In other words, people tend to underestimate both 
how much they learn from new information and the extent to which 
new information permits them to make correct judgments with greater 
confidence. To the extent that intelligence consumers manifest these 
same biases, they will tend to underrate the value to them of intelli- 
gence reporting. 


The Overseer’s Perspective 


An overseer, as the term is used here, is one who investigates 
intelligence performance by conducting a postmortem exainination of 
a high-profile intelligence failure. Such investigations are carried out 
by Congress, the Intelligence Community staff, and CIA or DI man- 
agement. For those outside the executive branch who do not regularly 
read the intelligence product, this sort of rc.rospective evaluation of 
known intelligence failures is a principal basis for judgments about 
the quality of intelligence analysis. 

A fundamental question posed in any postmortem investigation 
of intelligence failure is this: Given the information that was available 
at the time, should analysts have been able to foresee what was going 
to happen? Unbiased evaluation of intelligence performance depends 
upon the ability to provide an unbiased answer to this question.'” 


‘Experiments described in this section are reported in Baruch Fischhoff, “Hindsight # Foresight: 
The Effect of Outcome Knowledge on Judgment Under Uncertainty,” Journal of Experimental 
Psychology: Human Perception and Performance, |, 3 (1975), pp. 288-299. 


166 


Unfortunately, once an event has occurred, it is impossible to 
erase from our mind the knowledge of that event and reconstruct what 
our thought processes would have been at an earlier point in time. In 
reconstructing the past, there is a tendency toward determinism, 
toward thinking that what occurred was inevitable under the circum- 
stances and therefore predictable. In short, there is a tendency to 
believe analysts should have foreseen events that were, in fact, unfore- 
seeable on the basis of the information available at the time. 

The experiments reported in the following paragraphs tested the 
hypotheses that knowledge of an outcome increases the perceived 
inevitability of that outcome, and that people who are informed of the 
outcome are largely unaware that this information has changed their 
perceptions in this manner. 

A series of sub-experiments used brief (150-word) summaries of 
of these events was the struggle between the British and the Gurkhas 
in India in 1814. The four possible outcomes for this event were 1) 
British victory, 2) Gurkha victory, 3) military stelemate with no peace 
settlement, and 4) military stalemate with a pe.ce settlement. Five 
groups of 20 subjects each participated in each sub-experiment. One 
group received the 150-word description of the struggle between the 
British and the Gurkhas with no indication of the outcome. The other 
four groups received the identical description but with one sentence 
added to indicate the outcome of the struggle—a different outcome 
for each group. 

The subjects in all five groups were asked to estimate the !ikeli- 
hood of each of the four possible outcomes and to evaluate the rele- 
vance to their judgment of each datum in the event description. Those 
subjects who were informed of an outcome were placed in the same 
position as an overseer of intelligence analysis preparing a postmor- 
tem analysis of an intelligence failure. This person tries to assess the 
probability of an outcome based only on the information available 
before the outcome was known. The results are shown in Figure 18. 

The group not informed of any outcome judged the probability 
of Outcome | as 33.8 percent, while the group told that Outcome | 
was the actual outcome perceived the probability of this outcome as 
57.2 percent. The estimated probability was clearly influenced by 
knowledge of the outcome. Similarly, the control group with no out- 
come knowledge estimated the probability of Outcome 2 as 21.3 per- 
cent, while those informed that Outcome 2 was the actual outcome 
perceived it as having a 38.4 percent probability. 
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Figure 19 


Average Probabilities 
Experimental Assigned to Outcomes 


Groups 1 2 3 4 


Not Told Outcome 


Told Outcome 1 


Told Outcome 2 


Told Outcome 3 


Told Outcome 4 


An average of all estimated outcomes in six sub-experiments (a 
total of 2,188 estimates by 547 subjects) indicates that the knowledge 
or belief that one of four possibie outcomes has occurred approxi- 
mately doubles the perceived probability of that outcome as judged 
with hindsight as compared with foresight. 

The relevance that subjects attributed to any datum was also 
strongly influenced by which outcome, if any, they had been told was 
true. As Roberta Wohlstetter has written, “It is much easier after the 
fact to sort the relevant from the irrelevant signals. After the event, of 
course, a signal is always crystal clear. We can now see what disaster 
it was signaling since the disaster has occurred, but before the event it 
is obscure and pregnant with conflicting meanings.”'’ The fact that 
outcome knowledge automatically restructures a person's judgments 
about the relevance of available data is probably one reason it is so 
difficult to reconstruct how our thought processes were or would have 
been without this outcome knowledge. 


‘’ Roberta Wohlstetter, Pearl Harbor: Warning and Decision (Stanford, CA: Stanford University 
Press, 1962), p. 387. Cited by Fischhoff. 
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In several variations of this experiment, subjects were asked to 
respond as though they did not know the outcome, or as others would 
respond if they did not know the outcome. The results were little dif- 
ferent, indicating that subjects were largely unaware of how knowl- 
edge of the outcome affected their own perceptions. The experiment 
showed that subjects were unable to empathize with how others would 
judge these situations. Estimates of how others would interpret the 
data without knowing the outcome were virtually the same as the test 
subjects’ own retrospective interpretations. 

These results indicate that overseers conducting postmortem 
evaluations of what analysts should have been able to foresee, given 
the available information, will tend to perceive the outcome of that sit- 
uation as having been more predictable than was, in fact, the case. 
Because they are unable to reconstruct a state of mind that views the 
- situation only with foresight, not hindsight, overseers will tend to be - 
more critical of intelligence performance than is warranted. 


Discussion of Experiments 


Experiments that demonstrated these biases and their resistance 
to corrective action were conducted as part of a research program in 
decision analysis funded by the Defense Advanced Research Projects 
Agency. Unfortunately, the experimental subjects were students, not 
members of the Intelligence Community. There is, nonetheless, reason 
tu believe the results can be generalized to apply to the Intelligence 
Community. The experiments deal with basic human mental pro- 
cesses, and the results do seem consistent with personal experience in 
the Intelligence Community. In similar kinds of psychological tests, in 
which experts, including intelligence analysts, were used as test sub- 
jects, the experts showed the same pattern of responses as students. 

My own imperfect effort to replicate one of these experiments 
using intelligence analysts also supports the validity of the previous 
findings. To test the assertion that intelligence analysts normally over- 
estimate the accuracy of their past judgments, there are two necessary 
preconditions. First, analysts must make a series of estimates in quan- 
titative terms—that is, they inust say not just that a given occurrence 
is probable, but that there is, for example, a 75-percent chance of its 
occurrence. Second, it must be possible to make an unambiguous 
determination whether the estimated event did or did not occur. When 
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these two preconditions are present, one can go back and check the 
analysts’ recollections of their earlier estimates. Because CIA esti- 
mates are rarely stated in terms of quantitative probabilities, and 
because the occurrence of an estimated event within a specified time 
period often cannot be determined unambiguously, these precondi- 
tions are seldom met. 

I did, however, identify several analysts who, on two widely dif- 
fering subjects, had made quantitative estimates of the likelihood of 
events for which the subsequent outcome was clearly known. I went 
to these analysts and asked them to recall their earlier estimates. The 
conditions for this mini-experiment were far from ideal and the results 
were not clear-cut, but they did tend to support the conclusions druwn 
from the more extensive and systematic experiments described above. 

All this leads to the conclusion thet the three biases are found in 
Intelligence Community personnel as we!! as in the specific test sub- 
jects. In fact, one would expect the biases to be even greater in foreign 
affairs professionals whose careers and self-esteem depend upon the 
presumed accuracy of their judgments. 


Can We Overcome These Biases? 


Analysts tend to blame biased evaluations of intelligence perfor- 
mance at best on ignorance and at worst on self-interest and lack of 
objectivity. Both these factors may also be at work, but the experi- 
ments suggest the nature of human mental processes is also a princi- 
pal culprit. This is a more intractable cause than either ignorance or 
lack of objectivity. 

The self-interest of the experimental subjects was not at stake, yet 
they showd the same kinds of bias with which analysts are familiar. 
Moreover, in these experimental situations the biases were highly resis- 
tant to efforts to overcome them. Subjects were instructed to make esti- 
mates as if they did not already know the answer, but they were unable 
to do so. One set of test subjects was briefed specifically on the bias, 
Citing the results of previous experiments. This group was instructed to 
try to compensate for the bias, but it was unable to do so. Despite max- 
imum information and the best of intentions, the bias persisted. 

This intractability suggests the bias does indeed have its roots in 
the nature of uur mental processes. Anaiysts who try to recall a previ- 
ous estimate after learning the actual outcome of events, consumers 
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who think about how much a report has added to their knowledge, and 
overseers who judge whether analysts should have been able to avoid 
an intelligence failure, all have one thing in common. They are 
engaged in a mental process involving hindsight. They are trying to 
erase the impact of knowledge, so as to remember, reconstruct, or 
imagine the uncertainties they had or would have had about a subject 
prior to receipt of more or less definitive information. 

It appears, however, that the receipt of what is accepted as defin- 
itive or authoritative information causes an immediate but uncon- 
scious restructuring of a person's mental images to make them 
consistent with the new information. Once past perceptions have been 
restructured, it seems very difficult, if not impossible, to reconstruct 
accurately what one’s thought processes were or would have been 
before this restructuring. 

There is one procedure that may help to overcome these biases. 
It is to pose such questions as the following: Analysis should ask 
themselves, “If the opposite outcome had occurred, would I have been 
surprised?” Consumers should ask, “If this report had told me the 
opposite, would | have believed it?” And overseers should ask, “If the 
Opposite outcome had occurred, would it have been predictable given 
the information that was available?” These questions may help one 
recall or reconstruct the uncertainty that existed prior to learning the 
content of a report or the outcome of a situation. 

This method of overcoming the bias can be tested by readers of 
this chapter, especially those who believe it failed to tell them much 
they had not already known. If this chapter had reported that psycho- 
logical experiments show no consistent pattern of analysts overestimat- 
ing ‘he accuracy of their estimates, or of consumers underestimating 
the value of our product, would you have believed it? (Answer: Proba- 
bly not.) If it had reported that psychological experiments show these 
hiases to be caused only by self-interest and lack of objectivity, would 
you have believed this? (Answer: Probably yes.) And would you have 
believed it if this chapter had reported these biases can be overcome by 
a conscientious effort at objective evaluation? (Answer: Probably yes.) 

These questions may lead you, the reader, to recall the state of 
your knowlege or beliefs before reading this chapter. If so, the ques- 
tions will highlight what you learned here—namely, that significant 
biases in the evaluation of intelligence estimates are attributable to 
the nature of human mental processes, not just to self-interest and 
lack of objectivity, and that they are, therefore, exceedingly difficult 
to overcome. 
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PART IV—CONCLUSIONS 


Chapter 14 
Improving Intelligence Analysis 


This chapter offers a checklist for analysts—a summary of tips 
on how to navigate the minefield of problems identified in previous 
chapters. It also identifies steps that managers of intelligence analy- 
sis can take to help create an environment in which analytical excel- 
lence can flo::rish. 
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How can intelligence analysis be improved? That is the chal- 
lenge. A variety of traditional approaches are used in pursuing this 
goal: collecting more and better information for analysts to work with, 
changing the inanagement of the analytical process, increasing the 
number of analysts, providing language and area studies to improve 
analysts’ substantive expertise, revising employee selection and reten- 
tion criteria, improving report-writing skills, fine-tuning the relation- 
ship between intelligence analysts and intelligence consumers, and 
modifying the types of analytical products. 

Any of these measures may play an important role, but analysis 
is, above all, a mental process. Traditionally, analysts at all levels 
devote little attention to inproving how they think. To penetrate the 
heart and soul of the problem of improving analysis, it is necessary to 
better understand, influence, and guide the mental processes of ana- 
lysts themselves. 


Checklist for Analysts 


This checklist for analysts summarizes guidelines for maneuver- 
ing through the minefields encountered while proceeding through the 
analytical process. Following the guidelines will help analysts protect 
themselves from avoidable error and improve their chances of mak- 
ing the right calls. The discussion is organized around six key steps in 


the analytical process: defining the problem, generating hypotheses, 
collecting information, evaluating hypotheses, selecting the most 
likely hypothesis, and the ongoing monitoring of new information. 


Defining the Problem 

Start out by making certain you are asking—or being asked—the 
right questions. Do not hesitate to go back up the chain of command 
with a suggestion for doing something a little different from what was 
asked for. The policymaker who originated the requirement may not 
have thought through his or her needs, or the requirement may be 
somewhat garbled as it passes down through several echelons of man- 
agement. You may have a better understanding than the policymaker 
of what he or she needs, or should have, or what is possible to do. At 
the outset, also be sure your supervisor is aware of any tradeoff 
between quality of analysis and what you can accomplish within a 
specified time deadline. 


Generating Hypotheses 

Identify all the plausible hypotheses that need to be considered. 
Make a list of as many ideas as possible by consulting colleagu and 
outside experts. Do this in a brainstorming mode, suspendir: , _.- 
ment for as long as possible until all the ideas are out on the tauic. 

Then whittle the list down to a workable number of hypotheses 
for more detailed analysis. Frequently, one of these will be a decep- 
tion hypothesis—that another country or group is engaging in denial 
and deception to influence US perceptions or actions. 

At this stage, do not screen out reasonable hypotheses only 
because there is no evidence to support them. This applies in particu- 
lar to the deception hypothesis. If another country is concealing its 
intent through denial and deception, you should probably not expect 
to see evidence of it without completing a very careful analysis of this 
possibility. The deception hypothesis and other plausible hypotheses 
for which there may be no immediate evidence should be carried for- 
ward to the next stage of analysis until they can be carefully consid- 
ered and, if appropriate, rejected with good cause. 


Collecting Information 

Relying only on information that is automatically delivered to 
you will probably not solve all your analytical problems. To do the job 
right, it will probably be necessary to look elsewhere and dig for more 


information. Contact with the collectors, other Directorate of Opera- 
tions personnel, or first-cut analysts often yields additional informa- 
tion. Also check academic specialists, foreign newspapers, and 
specialized journals. 

Collect information to evaluate all the reasonable hypotheses, 
not just the one that seems most likely. Exploring alternative hypothe- 
ses that have not been seriously considered before often leads an ana- 
lyst into unexpected and unfamiliar territory. For example, evaluating 
the possibility of deception requires evaluating another country's or 
group's motives, opportunities, and means for denial and deception. 
This, in turn, may require understanding the strengths and weak- 
nesses of US human and technical collection capabilities. 

It is important to suspend judgment while information is being 
assembled on each of the hypotheses. It is easy to form impressions 
about a hypothesis on the basis of very little information, but hard to 
change an impression once it has taken root. If you find yourself 
thinking you already know the answer, ask yourself what would cause 
you to change your mind; then look for that information. 

Try to develop alternative hypotheses in order to determine if 
some alternative—when given a fair chance—might not be as compel- 
ling as your own preconceived view. Systematic development of an 
alternative hypothesis usually increases the perceived likelihood of 
that hypothesis. “A willingness to play with material from different 
angles and in the context of unpopular as well as popular hypotheses 
is an essential ingredient of a good detective, whether the end is the 
solution of a crime or an intelligence estimate.”'™ 


Evaluating Hypotheses 
Do not be misled by the fact that so much evidence supports 


your preconceived idea of which is the most likely hypothesis. That 
same evidence may be consistent with several different hypotheses. 
Focus on developing arguments against each hypothesis rather than 
trying to confirm hypotheses. In other words, pay particular attention 
to evidence or assumptions that suggest one or more hypotheses are 
less likely than the others. 

Recognize that your conclusions may be driven by assumptions 
that determine how you interpret the evidence rather than by the evi- 
dence itself. Especially critical are assumptions about what is in 


'™ Roberta Wohistetter, Pearl Harbor: Warning and Decision (Stanford: Stanford University Press, 
1962). p. 302. 
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another country’s national interest and how things are usually done in 
that country. Assumptions are fine as long as they are made explicit in 
your analysis and you analyze the sensitivity of your conclusions to 
those assumptions. Ask yourself, would different assumptions lead to 
a different interpretation of the evidence and different conclusions? 

Consider using the matrix format discussed in Chapter 8, “Anal- 
ysis of Competing Hypotheses,” to keep track of the evidence and 
how it relates to the various hypotheses. 

Guard against the various cognitive biases. Especially danger- 
ous are those biases that occur when you lack sufficient understand- 
ing of how a situation appears from another country’s point of view. 
Do not fill gaps in your knowledge by assuming that the other side is 
likely to act in a certain way because that is how the US Government 
would act, or other Americans would act, under similar circum- 
stances. 

Recognize that the US perception of another country’s national 
interest and decisionmaking processes often differs from how that 
country perceives its own in!-rests and how decisions are actually 
made in that country. In 198 J, for example, many analysts of Mid- 
die Eastern affairs clearly assumed that Iraq would demobilize part of 
its armed forces after the lengthy Iran-Iraq war so as to help rehabili- 
tate the Iraqi economy. They also believed Baghdad would see that 
attacking a neighboring Arab country would not be in Iraq’s best 
interest. We now know they were wrong. 

When making a judgment about what another country is likely to 
do, invest whatever time and effort are needed to consult with which- 
ever experts have the best understanding of what that country’s gov- 
ernment is actually thinking and how the decision is likely to be made. 

Do not assume that every foreign government action is based on 
a rational decision in pursuit of identified goals. Recognize that gov- 
ernment actions are sometimes best expiained as a product of bargain- 
ing among semi-independent bureaucratic entities, following standard 
operating procedures under inappropriate circumstances, unintended 
consequences, failure to follow orders, confusion, accident, or coinci- 
dence. 


Selecting the Most Likely Hypothesis 
Proceed by trying to reject hypotheses rather than confirm them. 


The most likely hypothesis is usually the one with the least evidence 
against it, not the one with the most evidence for it. 
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In presenting your conclusions, note all the reasonable hypothe- 
ses that were considered. Cite the arguments and evidence supporting 
your judgment, but also justify briefly why other alternatives were 
rejected or considered less likely. To avoid ambiguity, insert an odds 
ratio or probability range in parentheses after expressions of uncer- 
tainty in key judgments. 


Ongoing Monitoring 

In a rapidly changing, probabilistic world, analytical conclu- 
sions are always tentative. The situation may change, or it may remain 
unchanged while you receive new information that alters your under- 
s.anding of it. Specify things to look for that, if observed, would sug- 
gest a significant change in the probabilities. 

Pay particular attention to any feeling of surprise when new 
information does not fit your prior understanding. Consider whether 
this surprising information is consistent with an alternative hypothesis. 
A surprise or two, however small, may be the first clue that your 
understanding of what is happening requires some adjustment, is at 
best incomplete, or may be quite wrong. 


Management of Analysis 


The cognitive problems described in this book have implications 
for the management as well as the conduct of intelligence analysis. 
This concluding section looks at what managers of intelligence 
analysis can do to help create an organizational environment in which 
analytical excellence flourishes. These measures fall into four general 
categories: research, training, exposure to alternative mind-sets, and 
guiding analytical products. 


Support for Research 

Management should support research to gain a better under- 
standing of the cognitive processes involved in making intelligence 
judgments. There is a need for better understanding of the thinking 
skills involved in intelligence analysis, how to test job applicants for 
these skills, and how to train analysts to improve these skills. Analysts 
also need a fuller understanding of how cognitive limitations affect 
intelligence analysis and how to minimize their impact. They need 
simple tools and techniques to help protect themselves from avoid- 
able error. There is so much research to be done that it is difficult to 
know where to start. 
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Scholars selected for tours of duty in the Intelligence Community 
should include cognitive psychologists or other scholars of various back- 
grounds who are interested in studying the thinking processes of intelli- 
gence analysts. There should also be post-doctoral fellowships for 
promising scholars who could be encouraged to make a career of 
research in this field. Over time, this would contribute to building a better 
base of knowledge about how analysts do and/or should make analytical 
judgments and what tools or techniques can help them. 

Management should also support research on the mind-sets and 
implicit mental models of intelligence analysts. Because these mind- 
sets or models serve as a “screen” or “lens” through which analysts 
perceive foreign developments, research to determine the nature of 
this “lens” may contribute as much to accurate judgments as does 
research focused more directly on the foreign areas themselves. '* 


Training 


Most training of intelligence analysts is focused on organiza- 
tional procedures, writing style, and methodological techniques. Ana- 
lysts who write clearly are assumed to be thinking clearly. Yet it is 
quite possible to follow a faulty analytical process and write a clear 


and persuasive argument in support of an erroneous judgment. 

More training time should be devoted to the thinking and reason- 
ing processes involved in making intelligence judgments, and to the 
tools of the trade that are available to alleviate or compensate for the 
known cognitive problems encountered in analysis. This book is 
intended to support such training. 

Training will be more effective if supplemented with ongoing 
advice and assistance. An experienced coach who can monitor and 
guide ongoing performance is a valuable supplement to classroom 
instruction in many fields, probably including intelligence analysis. 
This is supposed to be the job of the branch chief or senior analyst, 
but these officers are often too busy responding to other pressing 
demands on their time. 


'*S Graham Allison's work on the Cuban missile crisis (Essence of Decision, Little, Brown & Co.. 
1971) is an example of what I have in mind. Allison identified three alternative assumptions about 
how governments work—the rational actor model, organizational process model, and bureaucratic 
politics model. He then showed how an analyst's implicit assumptions about the most appropriate 
model for analyzing a foreign government's behavior cause him or her to focus on different evi- 
dence and arrive at different conclusions. Another example is my own analysis of five alternative 
paths for making counterinte,!: “nce judgments in the controversial case of KGB defector Yuriy 
Nosenko. Richards J. Heuer, Jr., “Nosenko: Five Paths to Judgment.” Studies in Intelligence, Vol. 
31, No. 3 (Fall 1987), originally classified Secret but declassified and published in H. Bradford 
Westerfield, ed., Inside CIA's Private World: Declassified Articles from the Agency Internal Jour- 
nal 1955-1992 (New Haven: Yale University Press, 1995). 
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It would be worthwhile to consider how an analytical coaching 
staff might be formed to mentor new analysts or consult with analysts 
working particularly difficult issues. One possible model is the SCORE 
Organization that exists in many communities. SCORE stands for 
Senior Corps of Retired Executives. It is a national organization of 
retired executives who volunteer their time to counsel young entrepre- 
neurs starting their own businesses. It should be possible to form a 
small group of retired analysts who possess the skills and values that 
should be imparted to new analysts, and who would be willing to vol- 
unteer (or be hired) to come in several days a week to counsel junior 
analysts. 

New analysts could be required to read a specified set of books 
or articles relating to analysis, and to attend a half-day meeting once a 
month to discuss the reading and other experiences related to their 
development as analysts. A comparable voluntary program could be 
conducted for experienced analysts. This would help make analysts 
more conscious of the procedures they use in doing aualysis. In addi- 
tion to their educational value, the required readings and discussion 
would give analysts a common experience and vocabulary for commu- 
nicating with each other, and with management, about the problems of 
doing analysis. 

My suggestions for writings that would qualify for a mandatory 
reading program include: Robert Jervis’ Perception and Misperception 
in International Politics (Princeton University Press, 1977); Graham 
Allison’s Essence of Decision: Explaining the Cuban Missile Crisis 
(Little, Brown, 1971); Ernest May’s “Lessons” of the Past: The Use 
and Misuse of History in American Foreign Policy (Oxford University 
Press, 1973); Ephraim Kam’s, Surprise Attack (Harvard University 
Press, 1988); Richard Betts’ “Analysis, War and Decision: Why Intel- 
ligence Failures Are Inevitable,” World Politics, Vol. 31, No. 1 (Octo- 
ber 1978); Thomas Kuhn’s The Structure of Scientific Revolutions 
(University of Chicago Press, 1970); and Robin Hogarth’s Judgement 
and Choice (John Wiley, 1980). Although these were all written many 
years ago, they are classics of permanent value. Current analysts will 
doubtless have other works to recommend. CIA and Intelligence 
Community postmortem analyses of intelligence failure should also 
be part of the reading program. 

To facilitate institutional memory and learning, thorough postmor- 
tem analyses should be conducted on all significant intelligence fail- 
ures. Analytical (as distinct from collection) successes should also be 


studied. These analyses should be collated and maintained in a central 
location, available for review to identify the common characteristics of 
analytical failure and success. A meta-analysis of the causes and conse- 
quences of analytical success and failure should be widely distributed 
and used in training programs to heighten awareness of analytical prob- 
lems. 

To encourage learning from experience, even in the absence of a 
high-profile failure, management should require more frequent and 
systematic retrospective evaluation of analytical performance. One 
ought not generalize from any single instance of a correct or incorrect 
judgment, but a serics of related judgments that are, or are not, borne 
out by subsequent events can reveal the accuracy or inaccuracy of the 
analyst’s mental model. Obtaining systematic feedback on the accu- 
racy of past judgments is frequently difficult or impossible, especially 
in the political intelligence field. Political judgmenis are normally 
couched in imprecise terms and are generally conditional upon other 
developments. Even in retrospect, there are no objective criteria for 
evaluating the accuracy of most political intelligence judgments as 
they are presently written. 

In the economic and military fields, however, where estimates are 
frequently concerned with numerical quantities, systematic feedback 
on analytical performance is feasible. Retrospective evaluation should 
be standard procedure in those fields in which estimates are routinely 
updated at periodic intervals. The goal of learning from retrospective 
evaluation is achieved, however, only if it is accomplished as part of an 
objective search for improved understanding, not to identify scape- 
goats or assess blame. This requirement suggests that retrospective 
evaluation should be done routinely within the organizational unit that 
prepared the report, even at the cost of some loss of objectivity. 


Exposure to Alternative Mind-Sets 

The realities of bureaucratic life produce strong pressures for 
conformity. Management needs to make conscious efforts to ensure 
that well-reasoned competing views have the opportunity to surface 
within the Intelligence Community. Analysts need to enjoy a sense of 
security, so that partially developed new ideas may be expressed and 
bounced off others as sounding boards with minimal fear of criticism 
for deviating from established orthodoxy. 

Much of this book has dealt with ways of helping analysts 
remain more open to alternative views. Management can help by pro- 
moting the kinds of activities that confront analysts with alternative 


perspectives—consultation with outside experts, analytical debates, 
competitive analysis, devil’s advocates, gaming, and interdisciplinary 
brainstorming. 

Consultation with outside experts is especially important as a 
means of avoiding what Adm. David Jeremiah called the “everybody- 
thinks-like-us mindset” when making significant judgments that 
depend upon knowledge of a foreign culture. Intelligence analysts 
have often spent less time living in and absorbing the culture of the 
countries they are working on than outside experts on those countries. 
If analysts fail to understand the foreign culture, they will not see 
issues as the foreign government sees them. Instead, they may be 
inclined to mirror-image—that is, to assume that the other country’s 
leaders think like we do. The analyst assumes that the other country 
will do what we would do if we were in their shoes. 

Mirror-imaging is a common source of analytical error, and one 
that reportedly played a role in the Intelligence Community failure to 
warn of imminent Indian nuclear weapons testing in 1998. After lead- 
ing a US Government team that .nalyzed this episode, Adm. Jere- 
miah recommended more systematic use of outside expertise 
whenever there is a major transition that may lead to policy changes, 
such as the Hindu nationalists’ 1998 election victory and ascension to 
power in India.'* 

Pre-publication review of analytical reports offers another oppor- 
tunity to bring alternative perspectives to bear on an issue. Review 
procedures should explicitly question the mental model employed by 
the analyst in searching for and examining evidence. What assump- 
tions has the analyst made that are not discussed in the draft itself, but 
that underlie the principal judgments? What alternative hypotheses 
have been considered but rejected, and for what reason? What could 
cause the analyst to change his or her mind? 

Ideally, the review process should include analysts from other 
areas who are not specialists in the subject matter of the report. Ana- 
lysts within the same branch or division often share a similar mind- 
set. Past experience with review by analysts from other divisions or 
offices indicates that critical thinkers whose expertise is in other areas 
make a significant contribution. They often see things or ask questions 
that the author has not seen or asked. Because they are not so 
absorbed in the substance, they are better able to identify the assump- 
tions and assess the argumentation, internal consistency, logic, and 


'* Transcript of Adm. David Jeremiah’s news conference at CIA, 2 June 1998. 


relationship of the evidence to the conclusion. The reviewers also 
profit from the experience by learning standards for good analysis that 
are independent of the subject matter of the analysis. 


Guiding Analytical Products 

On key issues, management should reject most single-outcome 
analysis—that is, the single-minded focus on what the analyst believes 
is probably happening or most likely will happen. When we cannot 
afford to get it wrong, or when deception is a serious possibility, man- 
agement should consider mandating a systematic analytical process 
such as the one described in Chapter 8, “A’ sis of Competing 
Hypotheses.” Analysts should be required to iden.ify alternatives that 
were considered, justify why the alternatives are deemed less likely, 
and clearly express the degree of likelihood that events may not turn 
out as expected. 

Even if the analyst firmly believes the odds are, say, three-to-one 
against something happening, that leaves a 25-percent chance that it 
will occur. Making this explicit helps to better define the problem for 
the policymaker. Does that 25-percent chance merit some form of 
contingency planning? 

If the less likely hypothesis happens to be, for example, that a 
new Indian Government will actually follow through on its election 
campaign promise to conduct nuclear weapons testing, as recently 
occurred, even a 25-percent chance might be sufficient to put technical 
collection systems on increased alert. 

Verbal expressions of uncertainty—such as possible, probable, 
unlikely, may, and could—have long been recognized as sources of 
ambiguity and misunderstanding. By themselves, most verbal expres- 
sions of uncertainty are empty shells. The reader or listener fills them 
with meaning through the context in which they are used and what is 
already in the reader’s or listener’s mind about that subject. An intelli- 
gence consumer’s interpretation of imprecise probability judgments 
will always be biased in favor of consistency with what the reader 
already believes. That means the intelligence reports will be underval- 
ued and have little impact on the consumer’s judgment. This ambigu- 
ity can be especially troubling when dealing with low-probability, 
high-impact dangers against which policymakers may wish to make 
contingency plans. 
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Managers of intelligence analysis need to convey to analysts that 
it is okay to be uncertain, as long as they clearly inform readers of the 
degree of uncertainty, sources of uncertainty, and what milestones to 
watch for that might clarify the situation. Inserting odds ratios or 
numerical probability ranges in parentheses to clarify key points of an 
analysis should be standard practice. 

The likelihood of future surprises can be reduced if manage- 
ment assigns more resources to monitoring and analyzing seemingly 
low-probability events that will have a significant impact on US policy 
if they do occur. Analysts are often reluctant, on their own initiative, 
to devote time to studying things they do not believe will happen. This 
usually does not further an analyst’s career, although it can ruin a 
career when the unexpected does happen. Given the day-to-day pres- 
sures of current events, it is necessary for managers and analysts to 
clearly identify which unlikely but high-impact events need to be ana- 
lyzed and to allocate the resources to cover them. 

One guideline for identifying unlikely events that merit the spe- 
cific allocation of resources is to ask the following question: Are the 
chances of this happening, however small, sufficient that if policy- 
makers fully understood the risks, they might want to make contin- 
gency plans or take some form of preventive or preemptive action? If 
the answer is yes, resources should be committed to analyze even 
what appears to be an unlikely outcome. 

Managers of intelligence should support analyses that periodi- 
cally re-examine key problems from the ground up in order to avoid 
the pitfalls of the incremental approach. Receipt of information in 
small increments over time facilitates assimilation of this information 
to the analyst’s existing views. No one item of information may be 
sufficient to prompt the analyst to change a previous view. The cumu- 
lative message inherent in many pieces of information may be signifi- 
cant but is attenuated when this information is not examined as a 
whole. 

Finally, management should educate consumers concerning the 
limitations as well as the capabilities of intelligence analysis and 
should define a set of realistic expectations as a standard against 
which to judge analytical performance. 
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The Bottom Line 


Analysis can be improved! None of the measures discussed in 
this book will guarantee that accurate conclusions will be drawn from 
the incomplete and ambiguous information that intelligence analysts 
typically work with. Occasional intelligence failures must be 
expected. Collectively, however, the measures discussed here can cer- 
tainly improve the odds in the analysts’ favor. 
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